How to Outsmart the dreaded ‘Annual Disaster Recovery Test’

by | Nov 1, 2019 | Adamas News, Industry Focus

If there were not already enough tasks to complete and IT tickets to resolve, no one wants to carve away an entire day (or even take away a personal weekend) to shut all systems down – and ‘test’ the Disaster Recovery (DR) Process. The Disaster Recovery SOP serves to remind us that this day will come (each, and every year) and Auditors are sure to ask – ‘When was the last DR Test,… how did it go? Can we take a look at the evidence of DR testing…’
DR is the process of restoring mission critical business functions, systems, databases and IT infrastructure after a major negative event. A good DR Plan will prevent service interruptions and potential data loss. Preparing for what may be an unforeseeable event makes it almost impossible to mitigate all potential impacts. DR planning can build confidence in a reliable system design and framework for restoring operation, maintaining service lines, and protecting business critical data. All of which sounds excellent, but do you really want to commit your valuable business hours into the business of Disaster Recovery? Many do not. Compliance with the effective DR SOP sets the stage for the dreaded ‘Annual Disaster Recovery Test’.
With the wide variety of new hybrid enterprise cloud service providers and local server systems supported by a host of software tools that provide database analytics most organisations have ‘made their choice’ and locked into an IT system infrastructure that works for them. Based on the toolset and design parameters of a given system architecture we all want to see our RPO (Recovery Point Objective) and RTO (Recovery Time Objective) times drop from days to hours, and if hours to minutes. But regardless of the details of how long it takes to recover, and where the exact point of recovery resides, we find confidence in the security of the ‘Backup Server’.
There is a new paradigm that has ‘outsmarted’ the traditional model of testing the Disaster Recovery Plan. If not in full, at least in part, this new approach might re-invent the way we test, and the evidence we collect, to satisfy – and often exceed – the required limits of testing DR.
The simple trick is to look inside each organisation and find a repeated process (occurring quarterly, or monthly) that is divorced from the active day-to-day (‘live’) database. For example, most organisations with a Quality Assurance Department have an Annual Internal Audit Plan that requires the completion of a certain number of Internal Audits to conclude each quarter. Most often these Audits take a ‘look back’ (at the last 3-6 months) in an effort to measure Quality.
Two classic examples are Internal Audits of:

  • 1. Account Management of Software Applications;
  • 2. Onboarding Training Records of New Employees.

In an effort to outsmart the ‘Annual Disaster Recovery Test’ an organization could decide to utilize (exclusively) the Backup Server for all Internal Audits, rather than the primary (‘live’) server. Each Internal Audit now becomes a ‘test’ within itself that constitutes the accuracy and accessibility of the Backup Server, and the backup data.
So now when asked ‘Can we take a look at the most recent Disaster Recovery Test?’ This question could be answered by providing the Completed Internal Audit Schedule for the past year – as evidence of testing DR.
Many organisations that deliver study specific software as a service (SaaS) in a regulated environment, such as the Life Sciences, have made the decision to move all User Acceptance Testing (UAT) to the backup sever. Therefore, all UAT Test Scripts reference the URL of the Backup Server and once completed constitute a Disaster Recovery Test.
Depending on your organisation, find the right process (or two) to complete on the Backup Server and let the day-to-day completion of these tasks build your Disaster Recovery Test Repository. Then take a look at this year’s Calendar and on the scheduled day of the Disaster Recovery Test, find time for a holiday.

Mason Cox

Senior Consultant, Computer System Compliance
ADAMAS Consulting