Peoro's Odroid 🔗

This whole page is about the configuration peoro uses on his router/gateway odroids.

These odroids are meant to host all the vital services for a LAN (DHCP, DNS, natting etc) in a fancy way: routing some traffic through VPNs, blocking ads through custom DNS etc.

Running services 🔗

Check enabled systemd services with ls -lhR /etc/systemd/system/ | grep -- '->'.

When everything is configured, we should find the following running services:

dhcpd4-peoro, DHCP server.
dnsmasq, DNS server.
iptables, to automatically set iptables rules.
openvpn-client@allievi, openvpn-client@peori, our VPN clients.
serial-getty@ttyGS0: serial terminal.
lighthttpd or nginx for PI Hole's web interface.
rc-local, to run /etc/rc.local on boot.
rngd, a good random byte generator (install rng-tools).

The following should also be enabled, as they should be pre-configured with ALARM:

sshd, obviously.
systemd-networkd, handling our network configuration.
systemd-timesyncd, keeping our time synced.
getty@tty1, a terminal on TTY1.
amlogic, AMlogic HDMI Initialization. We can get rid of it.
remote-fs, meant as a dependency

Network configuration 🔗

To enable packet forwarding, create /etc/sysctl.d/nat.conf:

net.ipv4.ip_forward = 1

The enable natting through iptables.

iptables 🔗

Show the iptables configuration using:

sudo iptables -L -nv -t filter; echo; echo; sudo iptables -L -nv -t nat --line-numbers

We can save the iptables configuration, so that it's loaded when the iptables is started by running:

sudo iptables-save | sudo tee /etc/iptables/iptables.rules

In order to enable natting (i.e. replacing our IP into packets forwarded by us and then removing it from the responses) we can do:

iptables -t nat -A POSTROUTING -j MASQUERADE

And if we want to capture all the DNS requests (even the ones directed to remote DNS servers) to reply them ourselves, we can do:

iptables -t nat -A PREROUTING -p udp -m udp --dport 53 -j DNAT --to-destination 192.168.170.1

Where 192.168.170.1 is the local IP address.

OpenVPN 🔗

Allievi 🔗

remote allievi.sssup.it

# Auth
#auth-user-pass
ca   allievi/allievi-ca.crt
cert allievi/client.crt
key  allievi/client.key

# Net
dev tun0
client

# Technical details
proto udp
rport 1194
explicit-exit-notify

comp-lzo
auth SHA1       # Only for HMAC
ns-cert-type server

log-append /var/log/openvpn.allievi
verb 3

script-security 2
# adding this VPN as default gateway to table allievi and to the main table (with metric=20)
route-up        '/etc/openvpn/scripts/on-route-change.sh add 20'
route-pre-down  '/etc/openvpn/scripts/on-route-change.sh del 20'

peori 🔗

remote vpn.peori.space

# Auth
#auth-user-pass
ca   peori/ca.crt
cert peori/client.crt
key  peori/client.key

# Net
dev tun1
client

# Technical details
proto udp
rport 1194
explicit-exit-notify

comp-lzo
auth SHA1       # Only for HMAC
ns-cert-type server

log-append /var/log/openvpn.peori
verb 3

script-security 2
# adding this VPN as default gateway to table peori and to the main table (with metric=10)
route-up        '/etc/openvpn/scripts/on-route-change.sh add 10'
route-pre-down  '/etc/openvpn/scripts/on-route-change.sh del 10'

If you want to expose the odroid's LAN to peori...

Pick an even number for the client:
- 170: odroid-l77
- 172: odroid-w6
- 174: odroid-third
Use 192.168.NUMBER.0/24 for the new LAN.

On the VPN server, create peori/client-config/VPN_CLIENT_NAME:

# setting a static address for this client
ifconfig-push 172.16.13.NUMBER 172.16.13.(NUMBER+1)

# we're sending this client packets for it's /24 subnet
iroute 192.168.(NUMBER).0 255.255.255.0

/etc/openvpn/scripts/on-route-change.sh

🔗

This adds routes to and through this VPN. Check the routing rules section.

#!/bin/bash

set -x

C="$1"
METRIC="$2"

ip route $C $trusted_ip via $(ip route get $trusted_ip | awk "{print \$3; exit}")
ip route $C default     via $ifconfig_remote    dev $dev        table "${config%.*}"
ip route $C default     via $ifconfig_remote    dev $dev        table fallback  metric $METRIC

Remember to make this script executable.

Routing 🔗

We want to use rule based routing to send packets incoming from different sources to various destinations through specific routes.

Everything should reach some destinations (e.g. our LAN), only a few hosts should be allowed directly to Internet, while others should be routed to the Internet through our VPNs.

We can do this by using rule-based routing.

Overview 🔗

Let's declare the routing tables we'll use in /etc/iproute2/rt_tables:

255     local
254     main
253     default
0       unspec
#
# local
#
#1      inr.ruhep
500     router    # only used to reach the router's subnet
1000    direct    # direct internet connection
2000    allievi   # through allievi VPN
3000    peori     # through peori VPN
9999    fallback  # everybody falling here

When the kernel needs to decide where to route a packet, it uses the first table (ascending order) that matches the packet. We want to...

Let through all the packets directed to an enabled location (i.e. our local LAN, allievi and its network, peori and its network):

$ ip rule | grep main
32766:  from all lookup main                                         # everything matches
$ ip route show table main
193.205.80.47 via 192.168.254.254 dev eth0                           # allievi VPN server
10.2.84.0/22 via 172.16.0.25 dev tun0                                # allievi LAN
172.16.0.0/24 via 172.16.0.25 dev tun0                               # allievi VPN client network
172.16.0.25 dev tun0 proto kernel scope link src 172.16.0.26         # our allievi VPN client
51.15.55.198 via 192.168.254.254 dev eth0                            # peori VPN server
172.16.13.0/24 via 172.16.13.171 dev tun1                            # peori LAN
172.16.13.171 dev tun1 proto kernel scope link src 172.16.13.170     # our peori VPN client
192.168.170.0/24 dev eth0 proto kernel scope link src 192.168.170.1  # local LAN
192.168.254.0/24 dev eth0 proto kernel scope link src 192.168.254.1  # router LAN # TODO: shouldn't this go in the router table?

Let only our servers (with IPs in the range .1→.15) reach the router's LAN (in case they need to debug the router or whatever):

$ ip rule | grep router
50000:  from 192.168.170.0/28 lookup router  # only our servers match
$ ip route show table router
# TODO: shouldn't the router for 192.168.254.0/24 be here?

Let only localhost and a few cherrypicked hosts directly to the Internet through our router:

$ ip rule | grep direct
100000: from all iif lo lookup direct              # localhost (lo interface) matches
100000: from 192.168.170.33 lookup direct          # hanna-tablet matches
100000: from 192.168.170.4 lookup direct           # boox matches
$ ip route show table direct
default via 192.168.254.254 dev eth0 proto static  # local router address

Let anything else reach the Internet through our VPNs:

$ ip rule | grep fallback
999999: from all lookup fallback              # everything matches
$ ip route show table fallback
default via 172.16.0.25 dev tun0 metric 20    # internet through allievi
default via 172.16.13.171 dev tun1 metric 30  # internet through peori

Routing rules 🔗

We can modify routing rules with the ip rule command:

# packets from localhost (`lo` interface) go through the `direct` table
sudo ip rule add from all iif lo lookup direct priority 100000
# same for packets from 192.168.170.4
sudo ip rule add from 192.168.170.4 lookup direct priority 100000
# etc

We can install rc-local and place those commands in /etc/rc.local.

Or, a better solution, is to use systemd-networkd to manage the whole configuration.

Routes 🔗

To modify the routes we can use ip route ... table $TABLE ... and ip rule ... table $TABLE ... or do it through some configuration files...

Systemd-networkd configuration 🔗

Let's add all the rules and routes to /etc/systemd/network/eth0.network:

# Interface we're matching
[Match]
Name=eth0

# The IP address we set to the interface
[Network]
Address=192.168.170.1/24
Address=192.168.254.1/24


### Routing tables:

# `direct` table accepts packets from localhost
# ip rule add from all iif lo lookup direct priority 100000
[RoutingPolicyRule]
Table=1000
Priority=100000
IncomingInterface=lo  # fixed in https://github.com/systemd/systemd/issues/7210

# `router` table accepts trusted hosts (.0~.15)
# ip rule add from .0/28 lookup fallback priority 500000
[RoutingPolicyRule]
Table=500
Priority=50000
From=192.168.170.0/28

# `fallback` table accepts everything (with very low priority)
# ip rule add from all lookup fallback priority 999999
[RoutingPolicyRule]
Table=9999
Priority=999999


# adding a router through the router to `direct`
# ip route add default via .254 table direct dev eth0
[Route]
Table=1000
Gateway=192.168.170.254

# DISABLED, just for example...
# If our traffic to .254.0/24 (i.e. the actual router) needed to pass through .254 (i.e. access point between us and the router)
# ip route add .254.0/24 via .254 table router dev eth0
# [Route]
# Table=500
# Destination=192.168.254.0/24
# Gateway=192.168.170.254

The whole router table and everything about it, is just in case the router is on a different subnet than everything else.

This could be achieved if we use the router only as a modem (i.e. we don't use it neither as access point nor ethernet switch).

If that's not the case, ignore anything about it.

VPNs 🔗

We can also have alternative default routes (i.e. destination 0.0.0.0) through our VPNs. We decided to place these in the fallback routing table, which will be available to everybody.

A cleanish way to do this, is to add (and remove) these routes to the right table when the VPN connection goes up. Our OpenVPN configuration already runs a script that adds a route to the VPN network to the main routing table, and a default route (with custom metrix - i.e. priority) to the fallback table.

DHCP 🔗

Configure DHCPd4 in /etc/dhcpd.conf:

# enabling RFC 3442: classless static routes
option rfc3442-classless-static-routes code 121 = array of integer 8;
option ms-classless-static-routes code 249 = array of integer 8;

subnet 192.168.170.0 netmask 255.255.255.0 {
        # which IP addresses to assign
        range 192.168.170.100 192.168.170.250;

        # client's resolv.conf
        option domain-name-servers 192.168.170.1;
        option domain-search "l77.peoronet", "peoronet", "allievi.sssup.it";
        #option domain-name-servers 8.8.8.8;

        # setting the gateway through odroid
        option routers 192.168.170.1;
        #option routers 192.168.170.254;

        host tin {
                hardware ethernet 10:bf:48:69:fd:91;
                fixed-address 192.168.170.6;
        }
				
        # ...
}

Obviously fix all the IP addresses and domain names.

We'll need to start/enable the dhcpd4 service, but it's incompatible with the official Odroid kernel (3.16.57), since the latter doesn't support Ambient Capabilities. So create /etc/systemd/system/dhcpd4-peoro.service:

[Unit]
Description=IPv4 DHCP server
After=network.target network-online.target
Wants=network-online.target

[Service]
Type=forking
PIDFile=/run/dhcpd4.pid
ExecStart=/usr/bin/dhcpd -4 -q -user dhcp -cf /etc/dhcpd.conf -pf /run/dhcpd4.pid
ProtectSystem=full
ProtectHome=on
KillSignal=SIGINT
# We pull in network-online.target for a configured network connection.
# However this is not guaranteed to be the network connection our
# networks are configured for. So try to restart on failure with a delay
# of two seconds. Rate limiting kicks in after 12 seconds.
RestartSec=2s
Restart=on-failure
StartLimitInterval=12s

[Install]
WantedBy=multi-user.target

And start/enable that one instead.

DNS 🔗

We're using DNSMasq as DNS server.

/etc/dnsmasq.conf:

# Not reading /etc/hosts
no-hosts
# Use /etc/hosts.l77 instead
addn-hosts=/etc/hosts.l77

# Use the pihole blacklist
addn-hosts=/etc/pihole/gravity.list

# Ffilter out queries which the public DNS cannot answer, and which load the servers (especially the root servers) unnecessarily.
# these requests from bringing up the link unnecessarily.
# Never forward plain names (without a dot or domain part)
domain-needed
# Never forward addresses in the non-routed address spaces.
bogus-priv

# Don't read /etc/resolv.conf or any other file.
no-resolv

# Sending a request in parallel to all our DNS servers, and using the first result
all-servers
# Out DNS servers
server=/allievi.sssup.it/10.2.85.251
server=8.8.8.8
server=8.8.4.4

# Cache size
cache-size=10000

# For debugging purposes, log each DNS query as it passes through dnsmasq.
log-queries
log-facility=/run/log/pihole/pihole.log

# TTL for responses that come from /etc/hosts
local-ttl=60

# This allows it to continue functioning without being blocked by syslog, and allows syslog to use dnsmasq for DNS queries without risking deadlock
log-async

Pi Hole (ArchWiki) 🔗

Install pi-hole-server from AUR. Configure /etc/dnsmasq.conf and systemctl start pihole-FTL. Note that pi-hole will have installed its own fork of dnsmasq (called FTLDNS).

Append DBINTERVAL=60.0 to /etc/pihole/pihole-FTL.conf.

Run pihole -g to refresh the adblock list.

Disable (even temporarily) or re-enable pi-hole with:

pihole disable
pihole disable 5m  # 5 minutes
pihole enable

Web interface 🔗

TODO It sucks BTW D: It's in PHP...

Serial terminal 🔗

NOTE: if you want to use USB OTG while the Odroid is powered via a DC jack, you need to disconnect Odroid's jumper J1. Otherwise the odroid will overheat. (link). When J1 is connected, the odroid can be powered via the micro USB port.

Load the g_serial module on the raspberry to enable USB Gadget. We can load it permanently by creating /etc/modules-load.d/gserial.conf:

# Load g_serial, the gatget serial driver to use the micro USB port as a gadget device
g_serial

Once that's up, we'll have /dev/ttyGS0. We probably want to start a getty on it:

sudo systemctl enable --now serial-getty@ttyGS0

On the connected computer we can now access the odroid terminal (it should be /dev/ttyACM0). Run:

#!/bin/bash

# echo stty rows $LINES cols $COLUMNS

echo -e "Connecting to \e[31m/dev/ttyACM0\e[0m"
echo
echo -e "Exit pressing \e[33m~^d\e[0m"
echo
echo -e "After logging in, copy paste the following command:\e[0m"
echo -e "\e[32mstty rows $(tput lines) cols $(tput cols); set TERM $TERM; exec fish;\e[0m"
echo
cu -l "/dev/ttyACM0"

TODO 🔗

We need a script to keep the hostname↔IP mapping synchronized between DHCP and DNS.
We need to try and get an IP address from an existing DHCP server - if we receive no IP, only then we start our one. This way we'd be able to seamlessly switch between managed and unmanaged networks on the fly, making testing a lot easier.