SGI UV

The majority of open source bioinformatics applications in use today have been written for the Linux OS on x86_64 hardware, and this is a standard platform for the majority of Linux servers and HPC clusters. A commodity server available from most major vendors will typically support a maximum of 256GB or 512GB of RAM on the motherboard, which is adequate for the majority of applications. However there are algorithms in modern bioinformatics which require a very much larger memory footprint and where a standard server (or node in a cluster) is not a practical solution.

The SGI Altix UV range of system offers a large contiguous memory footprint within a single system image (SSI), which is of particular importance for memory intensive applications such as the ‘assembly’ of genome data output from high throughput genome sequencing instruments.

TGAC has three SGI UV systems. Two UV100 systems with 768 cores and 6TB of memory, and one UV2000 system with 2560 cores, 20TB of memory and 32 Xeon Phi modules.  The UV2000 has been divided into three hardware partitions, each of which runs a separate instance of Linux, and so they can be regarded as three separate systems.

System ModelUV100UV100UV2000(UV2000)(UV2000)UV300UV300
PartitionsPartition 1Partition 1Partition 1Partition 2Partition 3Partition 1Partition 1
System Nameuv1uv2uv2k1uv2k2uv2k3Ripley
DesignationDecommissionedProductionProductionProductionDevelopmentProductionProduction
CPU cores76876810241024512256256
Total RAM6TB6TB8TB8TB4TB12TB12TB
CPU modelIntel Xeon X7560, 2.27 GHzIntel Xeon E7-8837, 2.66 GHzIntel SandyBridge E5-4650L, 2.6 GHzIntel SandyBridge E5-4650L, 2.6 GHzIntel SandyBridge E5-4650L, 2.6 GHzIntel Haswell E7-8867V3, 2.5GHzIntel Haswell E7-8867V3, 2.5GHz
CPU cache24MB L324MB Smart Cache20MB Smart Cache20MB Smart Cache20MB Smart Cache45MB Smart Cache45MB Smart Cache
Offload processorsnonenonenonenone32 x Xeon Phi 5110P (Knights Corner 2250). 8GB RAM and 60 cores per Phi, 1.1GHznonenone
Blades484864644844
Sockets per blade222216 x 2 skt , 32 x 1 skt  *See Note 144
Cores per socket888881616
RAM per socket/blade64GB / 128GB64GB / 128GB64GB / 128GB64GB / 128GB64GB / (32x64GB , 16x128GB)  *See Note 1768GB / 3TB768GB / 3TB
RAM per core8GB8GB8GB8GB8GB48GB48GB
Operating SystemRHEL 6.2RHEL 6.4RHEL 6.4RHEL 6.4RHEL 6.4RHEL 7.1RHEL 7.1
Load SchedulerPBS Pro version 11.3PBS Pro version 11.2PBS Pro version 11.3PBS Pro version 11.3PBS Pro version 12.0SLURM v14.11SLURM v14.11
Scratch Storage20TB XFS88TB CXFS110TB CXFS  shared across all UV2000 partitions100TB S100 disk XFS

32TB NVMe FLASH
(16x 2TB P3700)
100TB S100 disk XFS

32TB NVMe FLASH
(16x 2TB P3700)

Note 1: 48 blades in total, 32 blades with a Phi installed, 16 without