Hail to Redundancy

Today I was literally saved by the bell... or rather saved by my RAID10.

When I bought my latest workstation I decided it would pay off to spare no expense, while trying to stay out of the "Game Machine" zone. One of the things that I decided I should have was hard disk redundancy. Backups and source control are must of course, but if my workstation crashes my problem is not losing the data. My problem is making an identically optimized workstation. I need about a week to do that.

  Just think about it: Reinstall Windows with Service Packs and a gazillion updates, Office with its own updates, Visual Studios and service packs, SDKs, WDKs, .NET frameworks of several versions and SPs, tools, etc, the endless list makes me shiver only to think of it. I have better things to do than looking at progress bars that pretend to be progressing thanks to some nice visuals.

So when time came I did what I thought best. I got an Intel WX58P workstation motherboard with hardware RAID and installed 4 identical 1TB hard disks in RAID10 configuration. In fact I got some "Enterprise Class" hard disk from Western Digital, just to be on the safe side.  

Today was pay-back day. Just 8 months after purchase and one of the disks in the RAID crashed. Here is how it looks like from the Intel Matrix Storage Console (click for details):

It took my hardware provider BARTEK (hail to them too) about 6 hours to come to my office with a replacement disk but it was the horroriest time (if there is such a word) that I've had in months!!! What if the currently not mirrored disk also crashed??? Oh my... I lose everything...

I tried to do some reasoning but it wouldn't be of much help. What is the possibility of two enterprise class hard disks crashing withing a 7 hour window? And especially the right two disks! But the one had already crashed, so it's history! Now I had 3 disks... What was the possibility of one crashing, especially the right one crashing? Especially given that another one of the same class and lot had already crashed! It's history of course but it can be indicative of problems in the whole lot... Probabilities were always confusing me...

And poor me, I had strong reasons not to shutdown the machine for 7 hours... Now that I think of it, it was not that wise, I should have shutdown to reduce the probability of an additional crash, but now it's history too...

Anyway the disk was replaced in time, the RAID was restored and now I am a happy blogger telling you how great it feels to have RAID save your ass(ets).

Enjoy!

Dimitrios Staikos

Comments

[adamo]

Well, in RAID-5 the probability of a second disk crashing while rebuilding is ~20% http://blog.postmaster.gr/2007/07/05/raid-scare/