This equipment analysed a wide range of biological and physical samples from unicellular organisms to tissues of both botanical and zoological nature and from nanoparticles to geological samples. It continues to be used by research personnel from many faculties and departments of NUS and industry.

Challenges to building the new facility

Prior to renovation, the Microscopy Facility experienced two major problems. The high relative humidity of the tropics caused instantaneous ice crystal formation on cryo-samples. This spurious ice contaminant coated the cryosamples and prevented structure determination in the electron microscope.

An equally serious problem was that a nearby building power and ground cables generated extremely high electromagnetic interference, EMI, which distorted the electron beam of all the TEMs and prevented them from reaching their full performance specifications.

JEOL and FEI engineers determined the sources of the EMI and contributed to the design of the Cryoelectron Microscopy Facility. As a result, the Facility was designed to support high-end cryoelectron microscopy by reducing humidity to 30% RH in the microscope and sample preparation rooms and by minimizing EMI by moving the power and ground lines for buildings S1, S2, S3, and S4 a suitable distance away from the Cryo Facility

After renovation, the Microscopy Facility was reorganized into the Cryoelectron Microscopy Facility by moving non-cryo-related functions to other sites including a scanning electron microscope to the TMSI and light microscopy support to building S1A.

Acquisition of the Titan Krios

The flagship instrument of the Facility is the FEI Titan Krios. Fully automated, the microscope lies within an outer box to give its iconic shape and shield the microscope from fluctuations in temperature and acoustic noise.

An autoloader feeds up to twelve samples into the specimen holder for imaging at liquid nitrogen temperatures. The microscope is operated remotely from a station located outside the microscope room. Images are collected by a 4K x 4K Gatan CCD camera after post-filtering through a Gatan GIF. Shortly after installation, the Krios was collecting high-resolution images of viruses and other biological specimens. This multi-million dollar instrument was purchased by funds from the Lee Hiok Kwee Fund.

History of the CBIS Computational Facility

Core Mission by founding architect, Alan Davis: CBIS’s computational core facility was created to store, process, analyse and visualise experimental data and images, as well as large scale computer modeling of cellular processes and data mining of complex biological data.

Challenges

Since the CBIS instruments can collect very large data sets approaching 1 TB in size, all IT infrastructure was designed to accommodate the movement and storage of large data sets during their acquisition, analysis and long term storage phases. Typical scientific laptops and workstations used by researchers, along with the local networks and storage systems, are not capable of handling this amount of data. In addition, the university general purpose computational facilities were unable to be used without invoking large time delays in data movement, restrictions in data set sizes, and offline batch computing analysis unsuitable for modern scientific research.

Facility Design

As CBIS moved to a new research space without any preexisting IT facilities, a new state of the art data centre was designed to accommodate the high performance computational, storage and networking capabilities planned for the facility. These IT resources were designed as a follow on to similar facilities designed by Alan Davis for Paul Matsudaira’s BioImaging Center and the Computational Systems Biology Institute at MIT in Cambridge, MA, although they were extensively updated after consultations with the CBIS researchers to accommodate their specific requirements and new instrumentation. CBIS researchers had designed a new Lightsheet Microscope capable of collecting data at very high data rates compared with traditional OMs, and would house multiple EMs compared to what was in use at MIT. This enhanced data collection capability meant all aspects of the IT resources would need updating for these increased data collection rates, storage capacity and data processing requirements.

IT Resources

The initial general purpose networking resource included 10 Gigabit Ethernet (GbE) connections not only within the data center but also to all microscope systems and individual workstations. This network has subsequently been upgraded to 40 GbE within the data center and to the EM facility.

The storage system initially consisted of a 100+ TB high performance storage array using Hierarchical Storage Management software to automatically manage the data. A dedicated high speed network was installed between the main storage array and the HPC servers to offload this data traffic from the slower general purpose network. This system was later expanded to 500 TBs of capacity with an archival tier. After many years in use, a new 1.2 BP storage array was installed utilizing InfiniBand (IB) networks as a replacement. This system is now in the process of being replaced by a 2 PB storage array to accommodate the ever expanding data storage requirements. Thus the storage capacity has increased 20-fold with a 5-fold increase in performance over the short lifetime of CBIS.

The compute servers initially included two multi-core SGI shared memory servers with > 100 TB of shared memory to facilitate applications that could utilize both many cores and large amounts of memory. Subsequently, this was augmented with two multi-node clusters, one with GPUs for applications that could use them for accelerated computational speed. The clusters are interconnected with a high speed IB network to facilitate applications that can run on multiple nodes. This has resulted in a 100-fold increase in computational performance since the start of CBIS.