Translator 2024 June 26, 8:00 | Category AI Artificial Intelligence, GPU, Chips. Author Jie An Chen

In the contemporary technology world, very few leaders can continuously seek breakthroughs and lead innovative trends like Elon Musk. Recently, Musk announced an unprecedented mega-project through his AI company xAI: the construction of a supercomputer with 100,000 NVIDIA H100 model GPUs, which will be on a scale far larger than any existing AI project.

How difficult is the scale of supercomputer calculation?

This ambitious idea is not only a revolution in hardware, but also symbolizes that AI is entering a new stage. In the past, large-scale computing resources were often viewed as the patent of academic research or national-level projects. Musk is now commercializing the concept of supercomputing. This supercomputer will handle a diverse range of complex tasks from language understanding to image analysis, providing ample computing power support for more advanced model training and data analysis.

From a technological standpoint, there are many challenges to installing such a large number of GPUs. First, there is the issue of physical size, how to properly configure such a scale of hardware devices, provide stable and efficient power supply, and solve the resulting heat dissipation problems, all of which require innovative system design and engineering solutions. Secondly, there is the challenge of network architecture; to achieve high-speed and efficient data transmission among 100,000 GPUs, advanced network technology and communication transmission are needed. Even how to release such a scale of computing power, and achieve system stability, will test the innovative strength of the development team.

However, once these challenges are overcome, the powerful computing capability will dramatically improve the efficiency of AI model training, allowing complex deep learning networks to be trained in a shorter amount of time. This means that AI systems can learn based on a more diverse and larger amount of data, and will gain higher accuracy and improve intelligence levels.

The demand for computational resources has exceeded the traditional infrastructure load

The use of a large number of GPUs is not just a pursuit of technological innovation, but also a practical choice based on the bottleneck faced by current AI development. As algorithms and models become increasingly complex, their demand for computational resources has far exceeded the load range of traditional infrastructure. For example, training the most advanced large language model GPT-4 requires a large number of GPUs for several months.

In order to support the effective operation of these large GPUs, significant inputs of energy and funds are also important. The price of these GPUs themselves can reach tens of thousands of dollars, plus the large amount of power supply required during operation, undoubtedly forming significant cost expenditures. In the long run, as technology advances, the demand for more powerful computing equipment will continue to grow. If companies can plan ahead and have a leading computing power infrastructure, they will gain dominance.

Supercomputing in the global technology race

xAI also faces a formidable opponent – Microsoft and OpenAI's collaboration relationship, especially the AI supercomputing center established on the Azure platform, has fully demonstrated their determination and layout. These supercomputing centers mainly provide the necessary computing power for increasingly complex AI models, and OpenAI's GPT-3 and its subsequent developed models are the basis for development and training on this platform. In comparison, Musk's grand vision is obviously taking a step further; he not only plans to build a single supercomputing center, but also hopes to create an AI computing power platform for the future through chips, energy, and innovation.

As more and more businesses and research institutions have access to the same level of computing resources, the democratization of AI technology will greatly increase. The future of innovation will no longer be overly concentrated in the hands of a few technology giants, and more small and medium-sized companies, academic teams, and individual developers will be able to participate, injecting a continuous stream of vitality into the entire industry. This unprecedented global computing power competition is thoroughly pushing the limits of technology and will reshape the entire landscape of the technology industry, making the future prospects of AI more expansive and promising.

• Elon Musk plans to build a supercomputer using NVIDIA’s semiconductor chips

（The source of the first image：shutterstock）