NVIDIA HGX

What is NVIDIA HGX

NVIDIA HGX is a specialized server platform from NVIDIA designed for high-performance computing (HPC), artificial intelligence (AI), deep learning, and cloud computing. It is a powerful hardware module with multiple (up to 8) NVIDIA GPUs connected by high-speed NVLink links via NVSwitch to efficiently train large language models (LLMs).

Key features and characteristics of NVIDIA HGX:

• GPU configuration: Up to 8 NVIDIA GPUs are standard, ranging from the A100 with 80 GB of HBM2 memory per GPU in base versions to the current H100/H200 in current generations.
• High Performance: With Ampere, Hopper, and Blackwell architectures, HGX delivers significant efficiency gains - in particular, HGX H200 servers can accelerate training of Transformer models (GPT, BERT, and others) up to ten times faster than previous generations.
• Interconnects: Utilize NVLink and NVSwitch to interconnect GPUs into a single compute cluster with very low latency and high throughput (e.g., up to 900 GB/s for H100 GPUs), critical for distributed learning and scalable cloud solutions.
• Flexibility and customization: The HGX platform is provided by NVIDIA as GPU modules (monolithic boards with up to 8 GPUs), while the choice of CPUs, storage, Infiniband and DPU adapters is left to the vendors (Supermicro, Dell, HPE, Lenovo, etc.), allowing you to create factory-built solutions with the optimal performance and features you need.
• Development history: The platform appeared in 2018 with HGX-2 based on Tesla V100 and has since evolved up to HGX B300 with Blackwell generation GPU.