MPI for Molecular Plant Physiology: Big Data in ZFS cluster

MPI Logo

The Max Planck Institute of Molecular Plant Physiology (MPI-MP) was founded in 1994. From originally only 16 employees the institute grew into a large institute, which now employs about 360 people who put their combined efforts into elucidating the secrets of plants.
In the first years the scientists performed their research in buildings on the campus of the University of Potsdam. In 1999 the construction activities on the Max Planck Campus were completed and the MPI-MP, together with the Max Planck Institute of Colloids and Interfaces and the Max Planck Institute for Gravitational Physics, moved into their new buildings. The three institutes share a central building with a lecture hall, seminar rooms, administration offices and a cafeteria. Apart from that, they are completely independent from each other.
EUROstor, a manufacturer of storage systems who’s main part of the product portfolio are server based systems, acting as flexible storage servers, tailor-made for the customers’ needs, has created a long-term business relationship with Max Planck Institute, supplying systems to this customer since 2007. Willing to provide their customer with the best solution they have decided for an Open-E product, not only because of the performance-price ratio, but also because the product has met all the strict requirements.

MPI Beitragsbild

The Mission

The Max Planck Institute of Molecular Plant Physiology was looking for a High Availability solution that is easily extensible and offers both NAS and iSCSI. A ZFS file system would be an additional asset as it offers an extra high degree of security against file system errors, which is particularly important for very large amounts of data. EUROstor, as a Platinum Partner of Open-E with many common and successful cluster solutions has made an obvious choice of ZFS- and Linux-based Open-E JovianDSS Data Storage Software to be implemented in the research institution’s storage environment.

The Solution

The Max Planck Institute of Molecular Plant Physiology was looking for a High Availability solution that is easily extensible and offers both NAS and iSCSI. A ZFS file system would be an additional asset as it offers an extra high degree of security against file system errors, which is particularly important for very large amounts of data. EUROstor, as a Platinum Partner of Open-E with many common and successful cluster solutions has made an obvious choice of ZFS- and Linux-based Open-E JovianDSS Data Storage Software to be implemented in the research institution’s storage environment.
The solution consists of a cluster node of two servers with dual Xeon processor (3.5 GHz), 256 GB RAM, 8 x 12 Gbit SAS on each of the JBODs. Four JBODs each with 44 x 3.5” disk slots, each equipped with 24 x 4TB nearline SAS disks are redundantly connected to both cluster nodes. The discs are merged into raidz2 vdevs with 2 disks per JBOD. This results in 6 + 2 redundancy (75% available gross capacity), which can result in a complete JBOD without interrupting availability.
There is 235 TiB of storage space available, added to this are write and read caches (ZIL, L2ARC) on mirrored SAS SSDs. Open-E JovianDSS provides continuous availability of all data and allows the institute to maintain all operations even if critical situations or disasters appear.
Now the system represents the central storage pool of the institute. It holds the storage space for a virtual server environment with about 100 machines, but also provides iSCSI storage targets for hardware-based file servers, which can be used for example provide the home directories for plenty of users, which was a must as from 16 employees the institute grew into 360 people from all over the world.

Open-E JovianDSS Logo

Now the system represents the central storage pool of the institute. It holds the storage space for a virtual server environment with about 100 machines, but also provides iSCSI storage targets for hardware-based file servers, which can be used for example provide the home directories for plenty of users, which was a must as from 16 employees the institute grew into 360 people from all over the world.

MPI Setup

Customer’s Voice

Carsten Bochan, Head of IT

”We chose this solution because of its best-in-class features. Open-E JovianDSS is based on the ZFS file system which is easy to configure and manage. High Availability provides us with the sense of security, as we can rest assured that our data is fully protected even in case of a serious error. We’ve been using this system already for a year and our experience of this product is very positive, as it is stable, robust and offers great performance.”

Weiterführende Links

Further References