• Home
• /
• KT Network

Dr Anjus George on the tangible scope of marriage between supercomputing and AI

Dr George throws light on how supercomputers are revolutionising AI research, accelerating innovation, and driving breakthroughs across industries

Two of the foremost technologies today complement and support each other to achieve great things. As artificial intelligence (AI) continues to reshape industries and drive innovation, the role of supercomputers in advancing this transformative technology has become increasingly vital. These high-performance computing systems are revolutionising AI research, enabling scientists and researchers to tackle complex problems at unprecedented speeds, and propelling progress across a range of sectors.

One of the names at the forefront of high-performance computing, Dr Anjus George, talked to us about the role that supercomputers are playing in furthering the potential of AI and other technologies. Currently working at Oak Ridge National Lab, Dr George has contributed to the development of the file system for some of the world's fastest supercomputers, including Frontier, which is currently the world's fastest supercomputer and is known for its impressive performance.

"Supercomputers are meticulously designed to process vast amounts of data and execute complex computations with exceptional speed, rendering them the perfect ally for AI applications," says Dr George. "Their powerful processors, massive storage capacities, and specialized architectures empower these computing mammoths to effortlessly tackle the intensive computational demands of AI algorithms and models, offering researchers the essential infrastructure to fully explore AI's boundless potential”, she elaborated.

Using supercomputers to explore artificial intelligence

“Thanks to advancements in GPU technology, it is now possible to create HPC (High-Performance Computing) clusters that can optimise AI and HPC software to perform complex industrial and scientific computation” — Dr George highlighted. AI has numerous applications like accelerating scientific discoveries, modelling earth's systems, climate change research, molecular modelling, and more. HPC can complement AI by providing the infrastructure for massive parallel computation and storage, allowing us to process large datasets and significantly increase the accuracy of AI models.

One significant area, as per Dr George, where supercomputers hit the spot, is in training deep neural networks. “Training these sophisticated AI models requires extensive computational power and enormous datasets. Supercomputers excel at parallel processing, dividing the workload across multiple processors, dramatically reducing training times from weeks to mere hours”, Anjus explained. This efficiency allows researchers to experiment with more extensive and more complex neural networks, leading to improved accuracy and performance in various AI applications, such as image recognition, natural language processing, and autonomous systems.

For instance, the ‘Frontier’ Supercomputer hosted at Oak Ridge National Laboratory — considered as the fastest and smartest AI-ready, high-performance computing (HPC) system — is one such system that sits at the intersection of HPC and AI. The Frontier has over 9400 AMD EPYC processors and over 37000 AMD instinct MI 250X GPUs.

It also uses a parallel distributed file system called 'Orion', which is based on Lustre technology that enables it to store and retrieve tens of petabytes of storage on hundreds of servers and achieve a throughput of more than a terabyte per second. Dr George has contributed to the development of Frontier and Lustre, a parallel file system utilised by tech teams in supercomputing worldwide. Her contributions have been valued within these projects and have been greatly helpful to the supercomputing community at large.

Supercomputers facilitate the exploration of AI algorithms and simulations, helping researchers gain a deeper understanding of the intricate workings of AI systems. These simulations allow scientists to simulate real-world scenarios, model complex phenomena, and test AI algorithms in a controlled environment. By leveraging supercomputers, researchers can accelerate the development and refinement of AI models, uncovering insights and solutions that can be applied to real-world challenges.

Hands-on implementation across industries

Dr George says, “The impact of supercomputers in AI extends beyond research laboratories. Industries ranging from healthcare and finance to manufacturing and energy are leveraging this computing power to drive innovation and optimise operations.”

In healthcare, supercomputers are aiding in drug discovery, analysing vast amounts of genetic data to identify potential treatments for diseases and accelerating the development of personalised medicine. In finance, these machines are used for high-frequency trading, risk assessment, and fraud detection, enhancing decision-making processes and minimising financial risks.

These machines are also seen powering advanced simulations and modelling in fields like weather forecasting, climate research, and aerospace engineering. Additionally, in the automotive and robotics industries, supercomputers play a crucial role in developing self-driving vehicles and intelligent robotic systems, enabling real-time perception, decision-making, and control.

However, in candid discourse, we may miss out on the innovations that lay the foundation of such hands-on implementations that we see at relatively tangible levels. Quicksilver, for instance, is one of Dr George’s projects which is a Lustre-based data management system that facilitates automated and scalable data movement across multiple tiers of Frontier’s storage system. This system enables AI applications to run efficiently by utilizing high performance storage layers, and is a model that exudes inspiration for the industry. Anjus, in her own right, is also a key contributor to characterizing ORNL’s next-generation supercomputer’s storage performance.

Road ahead

As the demand for AI continues to grow across industries, so does the need for more powerful supercomputers. As of 2023, efforts are already underway to develop and deploy even more advanced computing architectures, including quantum computers, to further enhance AI capabilities and unlock new frontiers. As accorded by Dr Anjus George, this convergence of supercomputing and AI promises to push the boundaries of innovation, opening doors to revolutionary applications and transforming industries on an unprecedented scale.

— Shagun Sharma is an independent business journalist.