search

unifi-utilities / unifios-utilities

A collection of enhancements for UnifiOS based devices
GNU General Public License v3.0
3.86k stars 415 forks source link

Unable to initiate a Wireguard connection to remote peer from the UDM-Pro #36

Closed dangle closed 3 years ago

dangle commented 3 years ago

I am unable to get wireguard running using the supplied instructions. The logs show an issue with the iptables-restore -n command that wg is running when setting up the connection.

To Reproduce Steps to reproduce the behavior:

  1. Follow the guide at https://github.com/boostchicken/udm-utilities/tree/master/wireguard-go
  2. Run podman exec -it wireguard wg
  3. There is no output.
  4. Run podman exec -it wireguard wg-quick up wg0

Expected behavior To see the connection to the Wireguard peer.

UDM Information

Additional context

[#] ip link add wg0 type wireguard
RTNETLINK answers: Not supported
[!] Missing WireGuard kernel module. Falling back to slow userspace implementation.
[#] wireguard-go wg0
WARNING WARNING WARNING WARNING WARNING WARNING WARNING
W                                                     G
W   You are running this software on a Linux kernel,  G
W   which is probably unnecessary and misguided. This G
W   is because the Linux kernel has built-in first    G
W   class support for WireGuard, and this support is  G
W   much more refined than this slower userspace      G
W   implementation. For more information on           G
W   installing the kernel module, please visit:       G
W           https://www.wireguard.com/install         G
W                                                     G
WARNING WARNING WARNING WARNING WARNING WARNING WARNING
INFO: (wg0) 2020/08/04 15:15:22 Starting wireguard-go version 0.0.20200320
[#] wg setconf wg0 /dev/fd/63
INFO: (wg0) 2020/08/04 15:15:22 Starting wireguard-go version 0.0.20200320
INFO: (wg0) 2020/08/04 15:15:22 Interface set up
INFO: (wg0) 2020/08/04 15:15:22 Device started
INFO: (wg0) 2020/08/04 15:15:22 UAPI listener started
[#] ip -4 address add 172.31.255.3/32 dev wg0
[#] ip link set mtu 1420 up dev wg0
[#] wg set wg0 fwmark 51820
[#] ip -4 route add 0.0.0.0/0 dev wg0 table 51820
[#] ip -4 rule add not fwmark 51820 table 51820
[#] ip -4 rule add table main suppress_prefixlength 0
[#] sysctl -q net.ipv4.conf.all.src_valid_mark=1
[#] iptables-restore -n
iptables-restore v1.8.4 (legacy): iptables-restore: unable to initialize table 'raw'

Error occurred at line: 1
Try `iptables-restore -h' or 'iptables-restore --help' for more information.
[#] ip -4 rule delete table 51820
[#] ip -4 rule delete table main suppress_prefixlength 0
[#] ip link delete dev wg0
INFO: (wg0) 2020/08/04 15:15:22 Interface set down
ERROR: (wg0) 2020/08/04 15:15:22 Failed to read packet from TUN device: read : file descriptor in bad state
INFO: (wg0) 2020/08/04 15:15:22 Device closing
INFO: (wg0) 2020/08/04 15:15:22 Shutting down
Error: non zero exit code: 2: OCI runtime error
boostchicken commented 3 years ago

Can you post your config? What is all that stuff going on with iptables restore?

boostchicken commented 3 years ago

Also, when the container launches it does wg-quick up /etc/wireguard/wg0.conf, you sure its not already running?

boostchicken commented 3 years ago

When this container is launched, its a one off. It gets deleted after every reboot. That is what the -rm flag does if you used my command. So it boots, brings up wg0, then waits for further commands. Do you have any custom PostUp or PostDown stuff in your wg0.conf?

dangle commented 3 years ago

It doesn't create the interface. If I exec into the container and run wg I get no output. wg0 doesn't appear in ip a, either.

My config is nothing special:

[Interface]
Address = 172.31.255.3/32
SaveConfig = true
PrivateKey = <private key>
DNS = 1.1.1.1

[Peer]
PublicKey = <server public key>
AllowedIPs = 0.0.0.0/0
Endpoint = <server IP>:51910
dangle commented 3 years ago

To be clear, when I say exec into the container, I mean I do this:

# podman exec -it wireguard bash
bash-5.0# wg
bash-5.0# ip a | grep wg0
bash-5.0# exit
boostchicken commented 3 years ago

So allowed-ip's 0.0.0.0 might bork everything. That is gonna tell the whole UDMP to send all traffic (without more specific routes) through wg0. Try a smaller subnet and see if that works.

My config

/ # more /etc/wireguard/wg0.conf
[Interface]
Address = 10.20.0.3/24
PrivateKey = xxxx
ListenPort = 51820

[Peer]
PublicKey = xxxxx
Endpoint = xxxx:54321
AllowedIPs = 10.20.0.0/24, 10.1.0.0/16, 10.2.0.0/16
[#] ip link add wg0 type wireguard
RTNETLINK answers: Not supported
[!] Missing WireGuard kernel module. Falling back to slow userspace implementation.
[#] wireguard-go wg0
WARNING WARNING WARNING WARNING WARNING WARNING WARNING
W                                                     G
W   You are running this software on a Linux kernel,  G
W   which is probably unnecessary and misguided. This G
W   is because the Linux kernel has built-in first    G
W   class support for WireGuard, and this support is  G
W   much more refined than this slower userspace      G
W   implementation. For more information on           G
W   installing the kernel module, please visit:       G
W           https://www.wireguard.com/install         G
W                                                     G
WARNING WARNING WARNING WARNING WARNING WARNING WARNING
INFO: (wg0) 2020/08/09 05:16:30 Starting wireguard-go version 0.0.20200320
[#] wg setconf wg0 /dev/fd/63
INFO: (wg0) 2020/08/09 05:16:30 Starting wireguard-go version 0.0.20200320
INFO: (wg0) 2020/08/09 05:16:30 Interface set up
INFO: (wg0) 2020/08/09 05:16:30 Device started
INFO: (wg0) 2020/08/09 05:16:30 UAPI listener started
[#] ip -4 address add 10.20.0.3/24 dev wg0
[#] ip link set mtu 1420 up dev wg0
[#] ip -4 route add 10.2.0.0/16 dev wg0
[#] ip -4 route add 10.1.0.0/16 dev wg0

I am not running the 6.0.x branch of the controller, but I am kind of at a loss why it is running any sort of IP tables rules? Maybe try ditching the DNS setting and Restricting allowed ips to see if that can at least get a tunnel working.

Also can you post your command line that is running the container and a podman inspect wireguard

boostchicken commented 3 years ago

On a side note, how is your address in a subnet that is a /32? No other addresses live in that subnet

Address = 172.31.255.3/32

Also, make sure the container is running with --privileged

dangle commented 3 years ago

It was the AllowedIPs = 0.0.0.0/0! I was hoping to route all traffic on a VLAN through the Wireguard gateway. I'm surprised it isn't using distinct network namespaces for the containers. I'll have to see how best to change this or just use an rPi instead.

On a side note, how is your address in a subnet that is a /32? No other addresses live in that subnet

Address = 172.31.255.3/32

Also, make sure the container is running with --privileged

I had actually just followed instructions in a guide that said to use /32. I had just assumed it was to limit the flow of traffic to be closer to a traditional client-server setup, but you're right, it really doesn't make sense. It doesn't seem to bother Wireguard though. I don't know enough about Wireguard to say why that would be though. More fun things to research.

Thank you very much!

boostchicken commented 3 years ago

@dangle it is using a distinct network ns for containers, just not the way I set it up. If you want your own netns, setup a macvlan like mode like I do for the dns containers. When you run --net=host it will run in the main network namespace. If you setup a macvlan, it will be in it's own ns.

Let me know if you need me to point you in the right direction, also if you get it working make sure to post examples here :)

  podman run -i -d --rm --net=wireguard --name wireguard2 --privileged \
    -v /mnt/data/wireguard:/etc/wireguard \
    -v /dev/net/tun:/dev/net/tun \
    -e LOG_LEVEL=info -e WG_COLOR_MODE=always \
    masipcat/wireguard-go

30-wireguard.conflist (symlinked to /etc/cni/net.d/30-wireguard.conflist)

{
  "cniVersion": "0.4.0",
  "name": "wireguard",
  "plugins": [
    {
      "type": "macvlan",
      "mode": "bridge",
      "master": "br5",
      "ipam": {
        "type": "static",
        "addresses": [
          {
            "address": "10.0.5.4/24",
            "gateway": "10.0.5.1"
          }
         ],
        "routes": [
          {"dst": "0.0.0.0/0"}
        ]
      }
    }
  ]
}
boostchicken commented 3 years ago

come to think of it, you wouldn't want to achieve this with routes. You would want to achieve it with iptables. I would use the PostUp command in wg0.conf to make some iptables rules that forward all your traffic from that vlan to wireguard tunnel, shouldn't be too hard

@dangle

boostchicken commented 3 years ago

Also some good ideas here

https://www.wireguard.com/netns/