New Cray is an AMD-powered, multi-petaflop Linux behemoth

30th May 2010
New Cray is  an AMD-powered,  multi-petaflop Linux  behemoth
The world's fastest computer?

Global supercomputer leader Cray has unveiled the company’s next high-end system – the Cray XE6 supercomputer -- at the annual Cray User Group (CUG) meeting in Edinburgh, Scotland, last week.

Combining the company’s new Gemini interconnect with AMD’s Opteron 6100  (8 or 12 core) processors – the same used in the company’s last supercomputer -- has enabled Cray offer high performance computer (HPC) users, a platform capable of touching multiple petaflops of performance and slapping together over 1 million computing cores. The operating system is Linux.

The new computer formerly codenamed Baker will be available only by end 2010, but Cray has booked orders from customers like the Korea Meteorological Administration and the United States Department of Energy’s National Energy Research Scientific Computing Centre (NERSC, the U.S. Air Force Research Laboratory, the Arctic Region Supercomputing Centre and the U.S Army Engineer Research and Development Centre as part of the Department of Defence’s High Performance Computing Modernization Program; the National Nuclear Security Administration (in a joint partnership with Los Alamos and Sandia National Laboratories), the Engineering and Physical Sciences Research Council (EPSRC), which is the managing agent for the High-End Computing Terascale Resource (HECToR) project located at Scotland’s University of Edinburgh, and the National Oceanic and Atmospheric Administration through a partnership with the Oak Ridge National Laboratory.

“We are proud to have already secured more than $200 million in contracts for this system and are even more excited about our opportunity to give researchers, scientists and engineers around the world a next-generation tool for solving next-generation problems”, says Peter Ungaro, president and CEO of Cray. The machine is reported to cost a cool $ 2 million.
At the heart of the Cray XE6 supercomputer is the new Gemini interconnect, which is designed to fundamentally change and significantly improve how Cray supercomputers move data across the system. Designed to support multi-core processors with a 100-fold improvement in messaging rates and a three-fold reduction in latency, the Gemini interconnect also includes hardware support for a global user address space. The Gemini interconnect is in part made possible by Cray’s participation in the Defense Advanced Research Projects Agency’s High Productivity Computing Systems program.

Fully upgradeable from a Cray XT5 and Cray XT6 system, the Cray XE6 supercomputer features additional enhancements such as an improved network resiliency, a mature scalable software ecosystem and the third version of the Cray Linux Environment, which was announced in April 2010.

According to the bi annual Top500 supercomputer rankings the Cray XT5 ‘Jaguar’ at the Oak Ridge National Laboratory was the first supercomputer to run a real-world scientific application at a sustained speed of more than one petaflops and is currently the world’s fastest computer, having posted a 1.75 petaflop/s performance speed running the Linpack benchmark. One petaflop/s refers to one quadrillion calculations per second.

XE6  key specifications:
Eight or 12-core 64-bit AMD Opteron 6100 Series processors; up to 192 per cabinet
64K L1 instruction cache, 64K L1 data cache, 512 KB L2 cache per processor core, 12 MB shared L3 cache
Memory: 32 GB or 64 GB registered ECC DDR3 SDRAM per compute node; Memory Bandwidth: 85.3 GB/s per compute node
Compute Cabinet: Cores : 1,536 or 2,304 processor cores per system cabinet
Peak Performance : 12.2 to 20.2 teraflops per system cabinet
Interconnect: 1 Gemini routing and communications ASIC per two compute nodes; 48 switch ports per Gemini chip, (160 GB/s internal switching capacity per chip)
Operating System: Cray Linux Environment (components include SUSE Linux SLES11, HSS and SMW software)
Power: 45-54.1 kW (45.9 – 55.2 kVA) per cabinet, depending on configuration
Product brochure:  

IDC's take on Cray's path to exascale computing: