• Login
  • |
  • Register
RSS Feed Print
Any experience with build your own storage?
Jim V
Posted: Saturday, May 21, 2011 4:16 AM
Joined: 3/4/2011
Posts: 1

We are starting to build up a small storage infrastructure ( ~ 100TB) in anticipation of our first sequencer. We have excellent, deep IT expertise. We have also received quotes from different vendors for storage.

Does anyone have actual experience with building your own storage and buying a complete solution from a vendor (such as Isilon)?


Chris Dagdiian
Posted: Wednesday, May 25, 2011 3:00 PM
Joined: 5/25/2011
Posts: 1


I have hands-on experience with DIY storage products as well as commercial kit from Isilon/Pansasas/DDN/Bluearc etc.  I've got a BlueArc Mercury system and a small Isilon cluster in my home office lab right now. I also hook these things up to NGS infrastructures all the time.


100TB is easy to do with DIY storage in the age of 2TB SATA disks and entry level storage chassis that hold 48-72 drives. I'm a fan of the NexSan SATABeast stuff when price constraints are the main driver as that company does a good "value storage" package without compromising on the critical stuff.  


It's much harder to go peta-scale or significantly over 100TB with home built solutions. It's completely possible, especially if you do it with a cluster or parallel filesystem on top but the administrative and operational burden climbs very quickly.


"Deep IT experience" may not translate into "we have storage administrators with free time right now" so my main advice is to characterize the operational burden for each of the systems you are configuring. If you have the staff resources available than there are no real technical challenges to rolling your own storage for NGS use, especially at the 100TB scale.


If you don't have staff resources available to manage the storage then the commercial scale-out NAS products make more sense. You'll pay more upfront but you'll get a system that requires far less work to maintain, operate and manage. It's not an urban legend when you hear stories about people managing 6+ petabytes of Isilon storage in their spare time - I've met those people.


The two main things to consider are operational burden and what the future size may look like. If you go much beyond 100TB with a "DIY" or homegrown solution it can get complicated and "brittle" quite quickly.








Posted: Tuesday, August 16, 2011 3:47 PM
Joined: 3/4/2011
Posts: 11

I completed setting up a ~100TB storage infrastructure for a customer (Govt lab) last year to support an Illumina GA IIx sequencer.  If you are thinking about this storage even before the sequencer has arrived, you are an exception but great! For most labs, I do not see a cost justification for high end storage solutions like Isilon / Bluearc. 2TB SATAs serve most purposes at a reasonable cost like Chris mentioned.

Consideration needs to be given to requirements such as scalability, data access, analysis, turnaround times, etc. Feel free to contact me if you need to talk.

Please login to post a new topic or reply to an existing post.
Privacy Policy|Terms of Use