Ooops

Just so you know: this page was imported from my old blog. Some pages were rather mangled in the process; my apologies if things don't quite look right.

Thanks to a brain failure on the part of some techies which left a couple of computers emitting smoke, UPSU.net went a little bit squiffy this morning...

UPSU.net offline message

The first I knew of it was a couple of minutes after the site went down and I received a nasty-looking error message about corrupted tables, database errors and passwords - first thoughts were "someone's managed to hack into the site - arghh!" and promptly had to deal with the panic of taking the whole site offline.

Thankfully, it wasn't anything as serious after all - the sudden shut-down of our database server meant one of the crucial lists of information - which tracks who's logged in - was corrupted and needed repair. This isn't the same as the list of members on UPSU.net, so losing the information would simply have meant anyone who was logged in to the site would have to log in again, at worst.

Things I've learnt from today:

  • We need a "big red button" which can be pressed to take UPSU.net offline and display a "Sorry, we broke it..." message.
  • And we need to make sure we know exactly how to press this "big red button".

Oh, and...

  • ... It's not worth having kittens over spilt pixels (or something similar...).

I'm not sure if we're out of the woods yet, as I understand there are a number of red-faced chaps running around an office somewhere in London trying to fix things before the end of the day (it is, after all, POETS day...), but we appear to be back up and running, so I can get on with finishing off the RSS system now. Huzzah...

Categories:

Most-mentioned in the blog