Disaster Recovery

Planning for a disaster IS rocket science. I started our Disaster Recovery Plan (DRP) & Business Continuity Plan (BCP) early in 1999, soon after I came to the company. It was a perfect time. Y2K was just around the corner & all they hype had the Suits worried. That meant resources (time!) to document the entire network. As I looked at all the pieces, I wrote out an outline of what to do in case each piece went down.

Y2K was such a non-event that no one was interested in taking the next step, though. Then 9/11 came along & suddenly it was hot again, but we were busy growing & the cost of a good DRP plan was too much for our stretched resources. Memories faded & so did the DRP plan - the little bit of it that I could do.

The CSX incident in Baltimore, a water main break & a fire in an underground room have all seriously disrupted services for other companies within blocks of us, but still we're too busy to look into it. It didn't happen to us or touch our wallets.

DRP for the basic network, parts of the phone & Internet services is something I can handle. BCP is something else though. How do we put a price tag on any kind of outage? No one is really out of a job when one system or another goes down for a day or two. Efficiency takes a hit - maybe. Most of us are so overwhelmed we'd enjoy a day without the phones or Internet to actually work undisturbed.

We could lose most systems singly for a day or two without it crippling the business. It's a hassle to lose the Internet & now that we have a couple of systems with an ASP, it can really put a crimp in our style. Of course, most people can go home & work from there on those systems in a real crunch. No Internet email, but there are still phones & faxes. Most of the systems have a work around like that.

The true disaster would be if the office space got wiped out. I looked into backups for that scenario & that's where the price tag becomes too high for anyone in management to be willing to pay. Reserving space & servers is expensive. A mirrored site is out of the question for us.
So the question became moot. I made up my own plan. We have some local sites where we could stash a server or two without a problem. Some spaces where we could put a skeleton staff. I've tried to make sure that we have good backups & copies of all our documentation, vendor agreements, install CD's & licensing. I keep them stashed in a safety deposit box & with a local backup vendor. Unfortunately, time to keep all this in order is not in my schedule, so I basically keep tossing new on top of old & occasionally get months behind.

No doubt about it, if the office space disappears, it's going to be ugly, but I think the business will survive. No one will be happy & doubtless I forgot a lot of critical items, but you do the best you can with what you have. Sometimes it doesn't seem like enough, but realistically, can we really afford to do much more?

Files, Backups & Junk

My backup tape ran out of room today. I wasn't terribly surprised. I've been trying to get management to notice the fact that files are growing at an astronomical rate & yet they still won't let me clean out any of the files from 10 years ago.

How do I get a handle on this problem? I've thought of a number of ideas & discarded most. I can't put limits on the amount of files the users keep or the size. Just reminding them they are over the limit on their email account is hassle enough. The Suits don't care about the size. They want everything to CYA & 'just in case'. I don't blame them - so do I.

I plan to pull all the files older than a certain date - trouble is, I've been asking for several years & still don't know what that date is. Three years sounds good to me. I'll create an 'Old' directory, that is read-only with the same file structure & put all the files I move off there. Then they can get to them quickly, if needed. Back them up a couple of times & they should always be available.

Clearing garbage won't help much or for long, though. Most of the files from more than a few years ago add up to about a third of the space taken up. Our old Word Perfect 6 files, although there are thousands of them, add up to only 3gb of space out of 150gb we're now using.

We have this catch-all directory where everyone has their personal files, files they share with others & all the group directories & files. Seven years ago, it was under 10gb. Since then we haven't done any serious cleaning, changing of the file structure or put any policies into effect to halt rampant growth. It isn't unusual for a single PowerPoint to take up 300mb. Everyone is scanning documents in now - we have workgroup scanners - & they're putting them in the same place as their regular Word documents.

Another option is a bigger backup. I'll have that ability in a few months, but again it is a short-term solution. Backing up that much data runs into the time crunch. There are NAS solutions, but getting the money for a larger backup was dicey.

I know this isn't an original problem & I've been looking at a lot of answers. Most of them are painful or expensive. All require that the Suits understand the issue - the hardest part - getting the executive buy-in.