My turn in the on-call rotation schedule came up again on Friday. So for the entire week I’m tethered to the pager, and must be able to respond after hours within 15 minutes. Needless to say, this limits how far I can venture from the house.
This is only my second time on-call, and during my first time, a hard drive on a firewall in Sterling Heights decided to croak. At 2am Saturday morning, just as our Leos party was starting to wind down. I logged 13 overtime hours that weekend alone, working on that problem onsite. My team lead sent me and a coworker to the data center there to replace the whole thing with a different type of firewall entirely. My co-workers assured me that such a thing was very rare, and only happened about once a year. Ok, I was willing to give them that.
Now we get to this weekend, five weeks later. Sunday morning, 5 AM, pager goes off. What has happened? The hard drive on a different firewall has just checked out and gone west. The backup drive was not configured. Luckily this time I didn’t have to go onsite. That job went to some poor schmuck in the Client Services department (who has a pretty sexy voice), but because Sun Microsystems support couldn’t get anyone onsite until 7 PM, I get stuck in the house in case they call early. They replace the hard drive, but no one has the CDs to install the operating system. I call my boss, who agrees that it can wait til the morning as long as no one calls to bitch about it.
So I finally get to sleep. Pager goes off at 4:30 AM Monday morning. I stagger to the laptop and check it out. Luckily it’s a false alarm, but I never really get back to sleep until I have to get up and go to work, where I spend literally the entire workday, plus an extra 1.5 hours, rebuilding the new hard drive.
Last night I go to German class, come home, eat, and start working on a bunch of firewall changes that have to be installed in the morning at 6 AM. They turn out to be more involved than originally anticipated, and I don’t get to bed until 1:30 AM. Guess what happens at 4:30 AM? Again, stagger out to the laptop, determine it’s a false alarm (same firewall as Monday morning), close the ticket, back to bed.
5:30 AM, pager goes off AGAIN. Same firewall. There’s nothing wrong with the box, it’s just lost it’s connection to the network. I ship the ticket to Network Services, so some poor slob on THEIR on-call list gets woken up. Have a nice freaking day.
Since the changes that had to be installed were for 6 AM, that counts as the start of my workday, so I leave an hour early. I get home, set up the laptop, and decide to take a nap since I’m so wiped out from the interrupted sleep of the last three days and fighting off the start of a cold.
When does the pager go off this time? 4:30 *PM*!
I am. Fucking. Cursed.