How to deal with OpenVPN Client Runtime Issues?

Once you have a VPN setup running in production, one of the main issues will become availability in one form or another. Earlier, I wrote about routing domains and how they can help you operate multiple OpenVPN client tunnels on the same gateway device.

Issues with Availability #

In my practical human trials, I have encountered issues mostly with availability. The most common issue in my experience has been that the openvpn process dies due to an error condition it cannot handle, such as socket availability or an error condition with the client-server communication that it cannot deal with in a controlled manner.

To this end, I first experimented with ifstated(8), but found out that it is not really the right tool for the task, since openvpn getting stuck is a rarer event than it exiting due to an error condition. Moreover, it remained unclear to me how to gracefully handle multiple independent processes and their state through the configuration. For example, the example config under

/etc/examples/ifstated.conf

was let’s say quite rudimentary. In the end I did learn a thing or two about ifstated, even if I end up quoting Thomas Edison:

I have not failed 10,000 times - I’ve successfully found 10,000 ways that will not work.

Bring out your dead! #

Since most of my challenges with availability have been related to a given VPN client exiting, I solved this problem with a simple shell script, which I run from cron every minute.

* * * * *       /etc/openvpn/scripts/resurrect.sh

It is a blunt instrument, but it gets the job done efficiently and mostly without fail. On average, the script has resurrected a dead tunnel four times a day with five concurrent tunnels operating 24/7 over the past three months.

#!/bin/ksh

TMP1=`mktemp`
TMP2=`mktemp`
TMP3=`mktemp`
PROC=`pgrep openvpn`

ls -1 /etc/openvpn/scripts | fgrep tun | sort > ${TMP1}

for p in ${PROC}
    do ps -p $p | fgrep tun | awk '{print $7}' >> ${TMP2}
done

sort ${TMP2} > ${TMP3}

DEAD=`comm -23 ${TMP1} ${TMP3}`

for D in $DEAD
    do logger -i "Resurrecting a dead tunnel: $D"
    rm -f /var/run/$D
    sh /etc/netstart $D
done

rm $TMP1 $TMP2 $TMP3

Don’t be a Stranger #

Since you have read up to this point, please do not hesitate to reach out to me if you have questions, comments, feedback or constructive criticism. You can find a contact email address in the site’s social links or maybe you want to give me a shout-out or follow on Twitter.