Detecting Outages

Millions of my fellow Texans were in crisis the week after Valentine’s day 2021. The weather was colder than it had been since Santa Anna marched his army to San Antonio, and the whole state was experiencing blackouts. But I was lucky. I had an exact record of precisely when the power shut off and was restored throughout this whole ordeal. How did this help me? It didn’t. But it was nice to know that my power outage notification system was working as intended.

I have an ODROID-HC2 computer in my closet that handles a variety of tasks. It is primarily a media server running Jellyfin so I can have a netflix-style interface for my personal movie collection. It also acts as an always-on syncthing node to keep the files on my desktop and laptop synchronized even when one of them is turned off. I figured that those functions were entirely too useful, and that it desperately needed at least one pointless task to carry out if it was going to be worthy of my network. Cue power outage detection.

The concept is fairly straightforward. The Odroid will make a connection to my webserver. If that connection breaks, that means there is a power outage, and the server should notify me. When the connection is restored, that means power has returned, and the server should notify me. The remote server is necessary because the Odroid would encounter some amount of difficulty when trying to tell me directly that it has no power.

The whole system was realized with 2 bash scripts. The first runs on the Odroid:

1#!/bin/bash
2
3while [ 1 ]
4do
5	ssh $MYREMOTESERVER "echo $(date +%s) > /tmp/dateping"
6	sleep 30
7done

This is about as simple as it gets. The script writes the current date and time to a file on my remote server called /tmp/dateping every 30 seconds. Is there a better way than opening a new ssh connection every 30 seconds? Probably, but I don’t care.

The second script runs on the remote server:

 1#!/bin/bash
 2poweron="true"
 3sleep 120
 4while [ 1 ]
 5do
 6    pingtime="$(cat /tmp/dateping)"
 7    sleep 2
 8    now="$(date +%s)"
 9    delta=$((now - pingtime))
10    if [ $delta -gt 80 ]; then
11        echo "poweron=$poweron"
12        if [ $poweron = "true" ]; then
13            poweron="false"
14            echo "Power outage at $(date)."
15            echo "Power outage at $(date)." | mail -aFrom:msg@cws.xyz 5558675309@tmomail.net
16        fi
17    else
18        if [ $poweron = "false" ]; then
19            poweron="true"
20            echo "Power restored at $(date)."
21            echo "Power restored at $(date)." | mail -aFrom:msg@cws.xyz 5558675309@tmomail.net
22        fi
23    fi
24    sleep 60
25done

This one is slightly more involved. Every minute, it checks if the current time is more than 80 seconds different than what is recorded in /tmp/dateping. If this is the case, the Odroid has missed sending at least 2 pings, meaning it has been disconnected for over a minute. From this we can conclude that the power is out and a message should be sent. At this point, we set the $poweron flag to "false". On the other hand, if the time difference is less than 80 seconds, we can conclude that the Odroid has power. In the case that the odroid has power and the $poweron flag is "false", we conclude that power has only just been restored so we send a message and reset $poweron.

Messages are sent using an SMS gateway, a system by which an email can be sent to a phone subscriber, who receives the message as a text. I use the GNU mail utility to send a message from a dummy address to [myphonenumber]@tmomail.net. Each cellular provider uses a different domain. If you’re in the US, you can find your provider in this table and try emailing a rude message to your phone number. Try it!

Finally, I would be remiss not to include a table of my data:

The longest power outage lasted over 13 hours, from 2pm to almost 4 in the morning - the coldest part of the day. My house relies on electricity for heat.