WG / UDMP Firewall Rules Interaction; UDMP Client List missing WG Peer

jhspyhard commented 2 years ago

I have been having a couple of issues with WireGuard which are probably configuration related, so please be gentle :).

Issue 1: I have set up VPN connectivity from my external smart phone into my UDMP's 192.168.26.0/24 VLAN, via WG. One of the problems that I running into, is that the WG wg0 interface seems to be side-stepping all of the other Ubiquiti LAN-IN type firewall rules that should apply to a machine sending requests originating from my 192.168.26.0/24 VLAN. For instance, from the connecting Peer device I am able to access some isolated hosts in my 192.168.5.0/24 VLAN which should be unable to reach each other.

My Wireguard configuration is as follows:

# SERVER SIDE CONFIG
[Interface]
Address = 192.168.26.1/32
ListenPort = 51820
PrivateKey = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
[Peer]
PublicKey = 9Qm7fmo8OSpDVBkONo/TydeLBKCx4gL7ZpaTszBKuVc=
AllowedIPs = 192.168.26.2/32

# CLIENT SIDE CONFIG
[Interface]
Address = 192.168.26.2/32
DNS = 192.168.5.32
PrivateKey = YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
[Peer]
AllowedIPs = 0.0.0.0/0, ::/0
Endpoint = 192.168.174.186:51820
PublicKey = 3tDUHW+3k/jUlLUTo2qtAXFEw6iBjN+8b31/cT3XMV8=

And info on the active wg0 connection:

# wg show wg0
interface: wg0
  public key: 3tDUHW+3k/jUlLUTo2qtAXFEw6iBjN+8b31/cT3XMV8=
  private key: (hidden)
  listening port: 51820

peer: 9Qm7fmo8OSpDVBkONo/TydeLBKCx4gL7ZpaTszBKuVc=
  endpoint: 192.168.174.144:39078
  allowed ips: 192.168.26.2/32
  latest handshake: 9 seconds ago
  transfer: 5.68 MiB received, 4.04 MiB sent

I had some PostUp and PostDown IPTables rules that I thought might be responsible for this expanded access, but having removed them completely the behavior seemed to continue.

Issue 2: The second problem may or may not be related to the first issue or maybe the UDMP's handling of non-DHCP set static IPs... I would have expected my clients connecting into the network via WG to show up in the Ubiquiti Network Controller's Clients list. They are NOT showing up for me. Do WG peers normally show up in that list and if so, any thoughts what I might need to change to get that working?

I've interacted a bit with you guys in r/Ubiquiti and you are awesome! Thanks so much for your time spent extending the UDMP with these sorts of useful functionalities, and your time in fielding networking-newbie posts like this.

peacey commented 2 years ago

Hi @jhspyhard,

With regards to the firewall rules, the problem is Ubiquiti only forwards interfaces it created to its rules. If you want to also forward the wireguard interface to the Ubiquiti rules, you can add custom rules to jump to the correct chains like this in SSH:
```
iptables -A FORWARD -i wg0 -j UBIOS_LAN_IN_USER
iptables -A FORWARD -o wg0 -j UBIOS_LAN_OUT_USER
iptables -A INPUT -i wg0 -j UBIOS_LAN_LOCAL_USER
```
Make sure to change wg0 to your interface name if different. This will add the jumps for LAN IN, LAN OUT, and LAN LOCAL for the wireguard interface, which will make it honour the rules.

You can also add it to PostUp/PreDown like so:
```
PostUp = iptables -A FORWARD -i %i -j UBIOS_LAN_IN_USER; iptables -A FORWARD -o %i -j UBIOS_LAN_OUT_USER; iptables -A INPUT -i %i -j UBIOS_LAN_LOCAL_USER
PreDown = iptables -D FORWARD -i %i -j UBIOS_LAN_IN_USER; iptables -D FORWARD -o %i -j UBIOS_LAN_OUT_USER; iptables -D INPUT -i %i -j UBIOS_LAN_LOCAL_USER
```
With regards to the client list, you will not be able to see WG clients on your Unifi dashboard. Ubiquiti most likely uses the neighbour protocol to find clients on the network. However, the WireGuard protocol does not support the neighbour protocol (nor any broadcast/multicast traffic). In addition, Ubiquiti might configure their detection to only work on the networks they created. So in any case, this is not supported.

jhspyhard commented 2 years ago

This definitely did the trick. Thank you so much for taking the time to reply with a great explanation of what was happening, and the steps to fix it!

peacey commented 2 years ago

No problem! I'll close this issue now since it's been answered, but if you have any more questions, please feel free to open another issue! Have a nice day.

jhspyhard commented 2 years ago

I had a bit of follow up on this ticket. I have been struggling to get my wireguard connection to survive reboots and I spent a bit of time troubleshooting and eventually ended up pulling the following error output from the wg-quick up wg0 command being run by the on_boot.d code:

[#] ip link add wg0 type wireguard
[#] wg setconf wg0 /dev/fd/63
[#] ip -4 address add 192.168.26.1/32 dev wg0
[#] ip link set mtu 1420 up dev wg0
[#] ip -4 route add 192.168.26.4/32 dev wg0
[#] ip -4 route add 192.168.26.3/32 dev wg0
[#] ip -4 route add 192.168.26.2/32 dev wg0
[#] iptables -A FORWARD -i wg0 -j UBIOS_LAN_IN_USER; iptables -A FORWARD -o wg0 -j UBIOS_LAN_OUT_USER; iptables -A INPUT -i wg0 -j UBIOS_LAN_LO                                                                                                                                                                              CAL_USER
iptables v1.6.2: Couldn't load target `UBIOS_LAN_IN_USER':No such file or directory

Try `iptables -h' or 'iptables --help' for more information.

It looks like I have some timing issues occurring where IPTables is trying to apply the Ubiquiti firewall chain rules to the wireguard connection, but the chains don't yet exist in this point of the boot-up processing.

I currently have my script set to a reasonably late priority 70_wireguard_connection.sh in the on_boot.d directory. Do you have any other suggestions in terms of delaying running the wg-quick up wg0 command until the UBIOS_LAN_IN_USER, UBIOS_LAN_OUT_USER and UBIOS_LAN_LOCAL_USER IPTables chains have come to exist?

peacey commented 2 years ago

Hi @jhspyhard,

On my UDMP it seems those chains exist when the script is run at boot even when it's the first script to run. Are you on the latest on-boot-script?

Anyhow maybe I'm just not encountering the timing issue. But just to be safe, you could add this snippet to your wireguard_connection.sh boot script before you run wg-quick.

for i in $(seq 300 -1 0); do
    (
    iptables -L UBIOS_LAN_IN_USER && \
    iptables -L UBIOS_LAN_OUT_USER && \
    iptables -L UBIOS_LAN_LOCAL_USER
    ) >/dev/null 2>&1 && break
    if [ "$i" = "0" ]; then
        echo "Timeout reached. Exiting."
        exit 1
    fi
    sleep 1
done

That will wait up to 300 seconds, checking if the chains exists every second and breaking the loop to continue with the boot script if it does exist. If after 300 seconds it still doesn't exist, the script will exit and NOT run wireguard.
You can change 300 to something higher if you want to wait even longer before timing out. 5 minutes should be plenty though. It will probably only take a couple seconds anyways unless something is wrong with the system.
Note because of how on_boot.d code is run, scripts with a later priority after this script will be delayed until this script finishes. So I would still put this script with late priority so it doesn't impact other scripts if it does wait for a while in some case.

jhspyhard commented 2 years ago

Hey @peacey

I think there must have been something not quite right with my install of udm-utilities. I reinstalled it, and it appears like now all of my scripts are consistently being run with no issues on boot.

I appreciate you spending a bit of your Saturday evening thinking through suggestions on things to try. Thank you - as always - and have a great rest of your weekend!

tusc / wireguard-kmod

WG / UDMP Firewall Rules Interaction; UDMP Client List missing WG Peer #34