Website availability

by Sean 30. July 2010 13:10

We had some problems with our website yesterday which resulted in it being unavailable between Wednesday afternoon and Thursday evening.  We are happy to say that the problem is fixed and that normal service is now resumed.  Please accept our apologies for any inconvenience this has caused you.

What was the Problem?

The problem was caused by a fault in one of our internet servers - our main website server in this case.  The fault caused the server to fail completely and we then had to rebuild the machine from scratch. 

We have plans in place to allow us to recover from disasters such as these, but the time it took us to recover from this one was almost 24 hours longer than we had anticipated.  In fact, we had the new server and website rebuilt and ready to go by the end of the afternoon on Wednesday. However, we had left one small, but important, thing out of our plans.

Why the Long Delay?

Our website is a secure website that is verified by a third-party security organisation called Entrust.  Basically, Entrust check our credentials and then issue us with a digital certificate that says we are who we say we are so that you can be sure with whom you are doing business.

image

Your internet browser will also show you that the website is secure and that it belongs to Electratest PIRform Ltd. in the website address:

SNAGHTML309d3e1

The problem was that we had forgotten to include this digital security certificate in our disaster recovery plans and we had to ask them to reissue our digital certificate to us.  To do this, they, quite reasonably, had to go through their security checks again.  Although they responded very quickly, these check are very thorough and take time to perform. 

Lessons Learnt

From when we were ready to go again on Wednesday evening, we sat around feeling like we had just rebuilt a car in record time and were then having to wait for someone to give us the key to start it up!  We have learnt two things from this:

  • Our existing disaster recover plans work pretty well.
  • We need to include our lovely certificate in those plans. 

As you can imagine, we have amended our disaster recovery plans and the digital certificate is now held very securely indeed!

Again, please accept our apologies for any inconvenience this caused you.

Tags: