A VPN (Virtual Private Network) tunnel creates an encrypted connection over the untrusted public WAN (Wide Area Network). You should use a VPN to protect yourself against nation state surveillance, but you can also leverage a VPN connection to access region-based restricted websites, shield your browsing activity from prying eyes on public Wi-Fi hotspots, prevent your Internet Service Provider (ISP) from applying Deep Packet Inspection (DPI) on your sessions, or to connect to your on-premises or cloud environment. This blog post will explain how to build an OpenVPN setup for all of the above.
We will use OpenVPN, easy-RSA OpenSSL, Pluggable Authentication Modules (PAM), OpenLDAP installed on Ubuntu 18.04.2 LTS together with Yubico YubiKey hardware security keys. Authentication will be done with X.509 Transport Layer Security (TLS 1.2) mutual authentication and Perfect Forward Secrecy (PFS) in combination with username, password and a YubiKey. Lastly, we will apply network segregation and host-based firewalling with iptables.
Please be advised that OpenSSL has been proven prone to several security vulnerabilities. OpenBSD's LibreSSL, an alternative to OpenSSL, provides a more solid foundation for your Public Key Infrastructure (PKI). Also be aware that whenever you create a VPN tunnel, you also create a possible way in for a malicious user. We will create a secure multi-user VPN setup as if we are fortifying during the siege of Damascus anno 1148.
Image source: Siege of Damascus, Chronique d’Ernoul 1148
Authentication process
Lets first talk about the authentication process which is shown in diagram 1:
1) A user tries to establish a session to the VPN server by resolving the VPN DNS alias. This A record alias resolves to a dedicated global routable IP-address bind to the OpenVPN services. The user presents its private key in the form of a X.509 certificate, along with its public key and the OpenVPN CA, username, password and YubiKey hardware token. Mutual authentication takes place with PFS.
2) X.509 mutual certificate based authentication takes place on the OpenVPN server. Next the OpenVPN server will check the LDAP username and the first 12 digits of the YubiKey One-Time Password (OTP) against its LDAP directory.
3) LDAP authentication results are sent to the OpenVPN server.
4) If the LDAP authentication is successful, the YubiKey OTP is validated against the global Yubico servers
5) Results are returned to the OpenVPN server.
6) When the user successfully authenticates all forms of authentication, a secure OpenVPN tunnel is established. The user is now able to access services on which the VPN server is present.
Diagram 1: Authentication process
Network interface settings
The OpenVPN box will have a total of three virtual Network Interface Cards (vNICs). The enp1s0 interface is used for incoming OpenVPN sessions. The enp2s0 interface is used for connecting to the LDAP server and the enp6s0 interface is used for management purposes such as SSH connections.
$ vi /etc/netplan/50-cloud-init.yaml
network:
version: 2
renderer: networkd
ethernets:
enp1s0:
dhcp4: no
addresses: [95.95.95.23/29]
gateway4: 95.95.95.21
nameservers:
addresses: [1.1.1.1,8.8.8.8]
enp2s0:
dhcp4: no
addresses: [192.168.70.2/24]
nameservers:
search: [cryptsus.local]
addresses: [192.168.1.253,192.168.1.254]
enp6s0:
dhcp4: no
addresses: [192.168.100.6/24]
nameservers:
search: [cryptsus.local]
addresses: [192.168.1.254,1.1.1.1]
Package installation
We install the following packages on the VPN server:
| Package | Purpose |
|---|---|
| openvpn | General server package |
| easy-rsa | Library to generate a PKI |
| libpam-yubico | YubiKey PAM module plugin |
Also make sure the local time is set correctly to prevent certificate time based errors:
$ rm /etc/localtime && ln -s /usr/share/zoneinfo/Europe/Amsterdam /etc/localtime
$ sudo add-apt-repository ppa:yubico/stable
$ apt-get update
$ apt-get upgrade
$ apt-get install openvpn easy-rsa libpam-yubico
Setup easy-RSA
We setup our PKI environment for our VPN server. Each client will get a certificate for their device. We configure the X.509 variables for easy-RSA:
$ sudo cp -r /usr/share/easy-rsa /etc/openvpn
$ mkdir /etc/openvpn/easy-rsa/keys
$ cd /etc/openvpn/easy-rsa
$ vi /etc/openvpn/easy-rsa/vars
#Exports
export EASY_RSA="/etc/openvpn/easy-rsa"
export OPENSSL="openssl"
export PKCS11TOOL="pkcs11-tool"
export GREP="grep"
export KEY_CONFIG="/etc/openvpn/easy-rsa/openssl-1.0.0.cnf"
export KEY_DIR="$EASY_RSA/keys"
echo NOTE: If you run ./clean-all, I will be doing a rm -rf on $KEY_DIR
#PKCS11 fixes
export PKCS11_MODULE_PATH="dummy"
export PKCS11_PIN="dummy"
#HD params size
export KEY_SIZE=2048
#In how many days should the root CA key expire
export CA_EXPIRE=3650
#In how many days should certificates expire
export KEY_EXPIRE=365
#PKI variables
export KEY_COUNTRY="NL"
export KEY_PROVINCE="Noord-Holland"
export KEY_CITY="Amsterdam"
export KEY_ORG="cryptsus.com"
export KEY_EMAIL="do-not-call-me@cryptsus.com"
export KEY_OU="IT Security Team"
Public-key Infrastructure (PKI)
OpenVPN supports bidirectional authentication based on certificates, meaning that the client must authenticate the server certificate and the server must authenticate the client certificate before mutual trust is established. Both server and client will authenticate the other by first verifying that the presented certificate was signed by the master certificate authority (CA). The following files will materialize out of the easy-RSA PKI tool:
| File name | File disclosure | Description |
|---|---|---|
| ca.key | private | The VPN root CA creates a tree of trust. The CA signs clients and the server certificates with its private key |
| ca.crt | public | The root CA is used to represent the identity of the CA itself. This file is signed by its own private key ca.key. This public component is merged into the client.ovpn file. |
| vpn.cryptvpn.com.key | private | The VPN server FQDN (Fully Qualified Domain Name) private key |
| vpn.cryptvpn.com.crt | public | VPN server FQDN public certificate |
| client1.key | private | Client private key. This private component is merged into the client.ovpn file. |
| client1.crt | public | Client certificate. This public component is merged into the client.ovpn file. |
| ta.key | private | Symmetric TLS authentication key. This key adds an additional layer of HMAC authentication to mitigate DoS attacks and attacks on the TLS stack. This component is also merged into the client.ovpn file. |
| dh2048.pem | public | Diffie-Hellman parameters will be used for TLS session in order to provide Perfect Forward Secrecy (PFS) |
| crl.pem | public | Registers revoked clients. The CRL allows compromised certificates to be selectively rejected without requiring that the entire PKI be rebuilt. |
Table 1: PKI files
After generating the PKI-files, we move them to the OpenVPN directory. We make sure the permissions on the private key files are not readable for other users.
$ ./clean-all
$ source ./vars
$ ./build-ca vpn-server nopass
$ ./build-key-server vpn.cryptvpn.com nopass
$ ./build-dh
$ openvpn --genkey --secret /etc/openvpn/easy-rsa/keys/ta.key
$ cp /etc/openvpn/easy-rsa/keys/{vpn.cryptvpn.com.crt,vpn.cryptvpn.com.key,ca.crt,ca.key,dh2048.pem,ta.key} /etc/openvpn
$ chmod 400 ca.crt ca.key vpn.cryptvpn.com.crt vpn.cryptvpn.com.key ta.key dh2048.pem
Create a new user
Next, we create a certificate for our clients. Repeat this process for each user and merge the client .crt and .key files into the client .ovpn file which is shown later on.
$ cd /etc/openvpn/easy-rsa
$ source ./vars
$ ./build-key [firstname-machine_name] nopass
LDAP and YubiKey configuration
We generate a shared symmetric key to validate OTP tokens with the YubiKey Cloud Validation Server or deploy your on-premises YubiKey Validation Server. Next, we configure the server-side PAM module to madatory verify LDAP credentials and YubiKey OTP's.
Now we extend the LDAP schema and register the first 12 digits every YubiKey One-Time Password (OTP) in the yubiKeyId LDAP attribute field. You can also choose to verify local users instead of AD/LDAP credentials if you have no centralized Identity and Access Manager (IAM) in place by leaving out the LDAP parameters.
$ ln -s /usr/lib/x86_64-linux-gnu/openvpn/plugins/openvpn-plugin-auth-pam.so /usr/lib/openvpn/openvpn-plugin-auth-pam.so
$ vi /etc/pam.d/openvpn
auth required pam_yubico.so ldap_uri=ldap://192.168.1.226 id=42282 key="api-key-here" yubi_attr=YubiKeyId ldapdn=DC=cryptsus,DC=local ldap_filter=(uid=%u) ldap_bind_user=CN=admin,DC=cryptsus,DC=local ldap_bind_password="super-secure-password-here"
account required pam_yubico.so
Server configuration
It is now time to configure our OpenVPN server with our preferred security settings. This config is shown in the below and is available on GitHub:
$ vi /etc/openvpn/server.conf
#Network settings
local 95.95.95.23
port 1194
proto udp
dev tun
tun-mtu 1500
#Certificate settings
ca /etc/openvpn/ca.crt
cert /etc/openvpn/vpn.cryptvpn.com.crt
key /etc/openvpn/vpn.cryptvpn.com.key
dh /etc/openvpn/dh2048.pem
tls-auth /etc/openvpn/ta.key 0
key-direction 0
crl-verify /etc/openvpn/crl.pem
#Crypto settings
tls-server
cipher AES-256-GCM
auth SHA256
tls-version-min 1.2
tls-cipher 'TLS-DHE-RSA-WITH-AES-256-GCM-SHA384:TLS-DHE-RSA-WITH-AES-256-CBC-SHA256'
ncp-ciphers 'AES-256-GCM:AES-128-GCM'
#Network settings
topology subnet
server 10.10.10.0 255.255.255.0
route 192.168.1.0 255.255.255.0
push "route 192.168.1.0 255.255.255.0"
push "redirect-gateway def1"
push "dhcp-option DNS 192.168.1.253"
push "dhcp-option DNS 192.168.1.254"
ifconfig-pool-persist /etc/openvpn/ipp.txt
#Miscellaneous
keepalive 10 120
auth-nocache
max-clients 10
explicit-exit-notify 1
reneg-sec 21600
#Permissions
user nobody
group nogroup
persist-key
persist-tun
#Logging
verb 6
mute 20
status /var/log/openvpn/openvpn.log
log /var/log/openvpn/openvpn.log
log-append /var/log/openvpn/openvpn.log
#YubiKey with LDAP integration
plugin /usr/lib/openvpn/openvpn-plugin-auth-pam.so openvpn
username-as-common-name
Client configuration
The client.ovpn needs to have the following settings configured in order to connect to the VPN server. Please note that the OpenVPN CA is placed inside the <ca> code block. The client certificates are placed under <cert> and <key>. The ta.key is placed inside the <tls-auth> code block. This means these certificates are in plain text inside this client.ovpn file.
client
remote vpn.cryptvpn.com 1194
proto udp
dev tun
tun-mtu 1500
resolv-retry infinite
nobind
persist-key
persist-tun
mute-replay-warnings
remote-cert-tls server
<ca>
</ca>
<cert>
</cert>
<key>
</key>
<tls-auth>
</tls-auth>
auth-nocache
key-direction 1
cipher AES-256-GCM
auth SHA256
verb 3
mute 20
auth-user-pass
OpenVPN daemon configuration
We will now use systemctl to start the OpenVPN daemon by passing the variable containing the name of our configuration file to the service unit. With systemd, we prefix the value of the @ symbol. The server value refers to the server.conf file.
$ service openvpn stop
$ systemctl start openvpn@server
$ sudo systemctl start openvpn@server
$ sudo systemctl is-active openvpn@server
Check connectivity
Next up we see if OpenVPN ACCEPTS sessions over UDP port 1194:
$ nc -uvz vpn.cryptvpn.com 1194
95.95.95.23: inverse host lookup failed: Unknown host
(UNKNOWN) [95.95.95.23] 1194 (openvpn) open
$ netstat -anlptu
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 619/systemd-resolve
tcp 0 0 192.168.100.6:22 0.0.0.0:* LISTEN 774/sshd
tcp 0 196 192.168.100.6:22 192.168.100.50:34131 ESTABLISHED 991/sshd: sudowoodoo
udp 0 0 127.0.0.53:53 0.0.0.0:* 619/systemd-resolve
udp 0 0 95.95.95.23:1194 0.0.0.0:* 1378/openvpn
Troubleshooting
Successful and failed and authentication attempts are logged in the auth.log file. Other system related information and errors can be found in the syslog and openvpn.log files:
$ cat /var/log/auth.log
$ cat /var/log/openvpn/openvpn.log
$ cat /var/log/syslog
VPN network routing configuration
We leverage the ip route command to configure gateway metrics, which are used to provide a connection to the LDAP server while maintaining a separate global routable IPv4 address for incoming VPN sessions. This means bidirectional traffic on UDP port 1194 is NATed through interface enp1s0, while interconnectivity between the VPN and the local network is preserved over a different network interface.
$ echo 1 > /proc/sys/net/ipv4/ip_forward
$ vi /etc/sysctl.conf
...
net.ipv4.ip_forward = 1
...
$ ip route add default via 95.95.95.23 dev enp1s0 metric 10
$ ip route add default via 192.168.70.1 dev enp2s0 metric 20
$ route add -net 192.168.1.0/24 gw 192.168.70.1
$ route -n
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 95.95.95.23 0.0.0.0 UG 0 0 0 enp1s0
0.0.0.0 95.95.95.23 0.0.0.0 UG 10 0 0 enp1s0
0.0.0.0 192.168.70.1 0.0.0.0 UG 20 0 0 enp2s0
10.10.10.0 0.0.0.0 255.255.255.0 U 0 0 0 tun0
192.168.1.0 192.168.70.1 255.255.255.0 UG 0 0 0 enp2s0
192.168.70.0 0.0.0.0 255.255.255.0 U 0 0 0 enp2s0
192.168.100.0 0.0.0.0 255.255.255.0 U 0 0 0 enp6s0
95.179.101.20 0.0.0.0 255.255.255.248 U 0 0 0 enp1s0
$ iptables -t nat -A POSTROUTING -s 10.10.10.0/24 -o enp1s0 -j MASQUERADE
$ iptables -t nat -A POSTROUTING -s 10.10.10.0/24 -p udp --dport 53 -d 192.168.1.0/24 -o enp2s0 -j MASQUERADE
$ iptables -t nat -A POSTROUTING -s 10.10.10.0/24 -p tcp --dport 443 -d 192.168.1.0/24 -o enp2s0 -j MASQUERADE
$ iptables -t nat -A POSTROUTING -s 10.10.10.0/24 -p tcp --dport 80 -d 192.168.1.0/24 -o enp2s0 -j MASQUERADE
$ iptables -t nat -A POSTROUTING -s 10.10.10.0/24 -p tcp --dport 22 -d 192.168.1.0/24 -o enp2s0 -j MASQUERADE
$ iptables -t nat -L -v
Chain POSTROUTING (policy ACCEPT 150 packets, 9664 bytes)
pkts bytes target prot opt in out source destination
212 13467 MASQUERADE all -- any enp1s0 10.10.10.0/24 anywhere
108 6897 MASQUERADE udp -- any enp2s0 10.10.10.0/24 192.168.1.0/24 udp dpt:domain
21 1344 MASQUERADE tcp -- any enp2s0 10.10.10.0/24 192.168.1.0/24 tcp dpt:https
0 0 MASQUERADE tcp -- any enp2s0 10.10.10.0/24 192.168.1.0/24 tcp dpt:http
0 0 MASQUERADE tcp -- any enp2s0 10.10.10.0/24 192.168.1.0/24 tcp dpt:ssh
System hardening
Make sure you harden your VPN box by configuring your firewall the right way. Also do not forget about regularly updating your box, do not run OpenVPN as root and harden your SSH daemon.
Think about an emergency policy where you would have to shutdown the VPN box due to a possible compromise. This is referred to as 'kill switch' which you can implement with iptables.
Lastly, look into a Hardware Security Module (HSM) to store your private keys via the OpenVPN PKCS #11 API. It is never a good idea to leave your PKI private keys in plain text on your VPN box.
