Building a DR site: Much more than a bunch of hardware

Recently I have gotten a lot of questions about building disaster recovery sites.   The one question I think I get the most is: “What is the most important part of building a disaster recovery site?”

Location, Location, Location

It’s easy to build a disaster recovery site attached to your second office, but if both offices are prone to natural disasters then you aren’t really building disaster recovery.  So what classifies disaster recovery anyway?  Why do people invest in disaster recovery?  Commonly people assume that your facility has to be hit by that of a disaster thus making it un-operational.  This really isn’t the case.  What if there is a malfunction with the fire suppression system thus rendering your datacenter inoperable?  What if the floor above you has a massive leak causing wide spread damage to your datacenter?  Most people don’t think of those things, but they can and will happen.  One 4th of July I stood in a datacenter trying to figure out how the AC unit seized up and started leaking through the vent.  Destroyed 4 servers, 1 of which was extremely critical to production.  Fortunately we had a server on hand as well as a backup, but I still was not as prepared for the situation as I should have been.

That’s when I started to view disaster recovery in a new light.  Disaster recovery should not be viewed as a worst-case-scenario but as a true backup or failover environment.  In the event that your SAN has a failure your DR site should have a fully working copy of the SAN.  The goal of DR should be less disaster and more completely seamless uninterrupted business.

Now what says you have to have your DR servers near the DR site?  With Cloud computing and datacenter management one could easily have their DR servers be in Boulder, CO with a DR facility in Kansas.

One might ask “well wouldn’t that defeat the purpose of having a DR facility?”  Sure having your office in Kansas where Tornado’s can knock your facility offline can seem a little futile to the DR concept, however, what if you already have a second facility there?  Are you going to move that one or invest in a third simply to facilitate a disaster recovery?  The chances of both your facilities, if strategically placed geographically, being knocked out is rare --- but that doesn’t mean you should invest your technology there.

When building a disaster recovery site you should talk to a Network Architect or consultant to see what is best for your business model.  There are lots of things to consider when building a DR site and we’ve probably run into them.

So when planning a DR site what are some things you should take into proper consideration:

1) Location.  Is your location prone to disaster?  Can your business model lend itself to having servers in a managed datacenter vs. onsite?  These are questions you should ask yourself when picking the location appropriate to your business.

2) Plan.  Even though we’re sure you’re going to make it through the disaster, you still need a well organized plan.  Keep in mind, this is supposed to go off problem free when failed over and require little to no human intervention

3) Hardware consideration.  Though it is simply a failover/backup environment, you should still give it the same attention you give corporate.  If you are required to be on the system for any length of time you do not want to have issues arise that it cannot handle.  Far too often people paint themselves in a corner because they “never expected to be on DR this long.”

4) Test. Having a plan and DR site is great, but you have to actually test it.  You have a plan, you’ve designed the best environment for your organization, but now it’s time to test/train your staff.  When dealing with a disaster people don’t usually keep a level head and stay calm.  It’s good to go through the drill.

With these steps and proper planning by a Network Architect then you can design a very robust and effective DR site.