Disaster Management Plans
Over the weekend, the big story in the news was all about the IT system failure at British Airways. The CEO of BA has released a statement that the systems crashed due to a short power surge knocking out the main servers, unfortunately for the company the back-up systems then failed to take over. This resulted in over 75,000 customers being affected and flights grounded around the world.
When the story first broke, BA did not release much information to the public, no doubt as the Directors were scrambling to find out what had gone so badly wrong. However, this resulted in various speculation such as systems being hacked, no back up servers in place or that this was due to cost cutting by out sourcing IT Support. It would appear, that none of these were the case.
Many people in the IT industry are watching this story closely to understand how this situation happened. With many questions being asked, including:
- Why did the back-up servers not take over?
- Are the back-up systems in the same location as the main servers?
- Are the communication systems such as website and mobile app in the same location?
- What was the Disaster Management Plan?
Irrespective of your IT Support being in house or outsourced these questions should not be being asked. Here at MPR IT, we support a wide range of customers for IT, on both a daily basis and, also with larger projects such as server/network installations and server hosting. With all our customers, we hold the same approach, no matter the size of your company – you need to be prepared for any eventuality. Whilst we have not had to deal with an incident the size of BA’s problems at the weekend we have had to deal with various situations where a company server has gone off line. These include a flood in an office, or a system hack. Essentially the plan is the same – get the company back up and running in the shortest time possible and preferably without their customers ever knowing there was an issue and with the least cost involved. I think we can quite safely say this is going to be very expensive for BA.
Disaster Management Recovery plans are one of the most important things that your business can put in place. The aim of a DMR is to ensure that your business continues smoothly with little or no disruption, and your company needs will differ from that of any other company. The main points that you need to consider are:
- Is your data/server backed up daily?
- Is this back up stored in a secure separate location to the main server?
- Can your customers still contact you?
- Can your staff continue to carry out their duties?
- Can your operation continue and will your customers be affected?
We have written blogs in the past about important it is to have a Disaster Management Plan in place, however the events this weekend have shown that it is not just in the case of fire, flood or bad weather that your company needs to plan, in advance for. In the case of BA it was apparently due to a power surge, that the systems failed. Not as many speculated in the first few hours, that it was due to out sourced IT Support or being hacked with a virus. If you and your company would like to discuss your current Disaster Management Plans please do not hesitate to contact us.Read more
Emergencies can happen in any office, as one of our clients found out one weekend.
One of our long standing clients, Russell Laboratories, has been receiving support from MPR IT for a number of years now. The package we provide includes our fully inclusive Server Support package, as well as a number of larger projects including server replacements, network cabling and CCTV. MPR IT and Russell Laboratories have a close working relationship and as with all our customers we meet regularly to discuss the support we provide, and help to identify any areas that require attention.
Russell Laboratories are a leading provider of advanced plating and finishing services to the electronics, aerospace and allied industries. Based locally in Ashford the company has been in operation for 40 years. The decision to outsource IT support was made to allow the existing staff, at Russell Laboratories to concentrate on their own main roles within the business. As well as having the knowledge of a fixed price contract for ongoing daily support and security that any additional work required would be carried out by experienced engineers who fully understand the network.
As part of the inclusive server support, our systems monitor the servers on site 24 hours a day, 365 days a year, alerting our team to any issues with the systems. This means that even at weekends or over holiday periods, Russell Laboratories are safe in the knowledge that should emergencies happen they have no costly invoices for out of hour’s support.
One Saturday morning, our out of hour’s team were alerted to an issue with the server at Russell Laboratories. Our on call engineer contacted them to discuss this and it was then discovered that a water leak had occurred and the server had been damaged by water. With the knowledge and expertise of our engineer, within a short period of time we were able to transfer email access to Office 365 enabling the flow of email to continue uninterrupted. After a quick site visit to identify the equipment affected, new hardware was ordered to arrive the next working day ready for installation. As part of the support package we provide, Servers are backed up daily by MPR IT and with the data from the Friday night incident it meant that no data had been lost. The team at Russell Laboratories lost no data and they were able to continue working, with access to their emails and the total downtime from the first notification of the incident was 48 hours over the weekend.Read more
Canterbury Archaeological Trust, one of the UK most famous Archaeology firms based in Kent, turned to MPR IT over 5 years ago looking for an outsourced IT company. They required a reliable company to provide day to day support, Office 365, Anti-virus and also support on any larger projects that may require IT services.
Included in the support that we offer to the Trust is an all-inclusive server support and AVG anti-virus software. Ensuring the charity is safe in the knowledge that the systems are monitored and support available 365 days a year, all for a set cost per month. No unexpected bills for when something goes awry on a weekend or holiday period.
This support became invaluable one weekend when something went wrong on the network. As 24 hours support was in place, an out of hour’s call was made to MPR IT. With the ability to remotely look at the network our engineer quickly identified that malware had in fact infiltrated the system, and he quickly made the decision to make his way to the site.
Within an hour of the initial call out the network on site had been shut down and all machines physically unplugged to avoid any further damage occurring. This is key with any malware attack, the sooner the network is detached from the outside world the quicker it will stop from spreading. Over the next few hours, the infected file was identified and removed whilst the remaining files were scanned. The network is backed up on a regular basis,which is something we recommend to all our clients. This meant that when restoring the network it was easy to revert to the most recent back up and limited data lost, especially as this occurred out of hours meaning that there had been limited access since the back up the previous evening.
What Canterbury Archaeological Trust have to say about MPR IT
Just a quick note to thank you and the team – and especially David and Mitch – for the fantastic response to the weekend viral attack on our computer systems, and for your sound advice in the first place which resulted in building such a resilient set-up. From your first response on Monday, which limited the damage, to the speedy and efficient disaster recovery, we could not have asked for more. Several of our staff have commented on their amazement at the speed with which you got us back up and running. Brilliant job, well done!
Personally, I can’t help contrasting it with what happened a few years back when the Raid Array blew, that really was a disaster. We’ve come a very long way since then, thanks to MPR-IT.
Commercial Operations Director
Canterbury Archaeological Trust
For more information on how we can support your business please do not hesitate to contact us and discuss your requirements.Read more