search

qdm12 / gluetun

VPN client in a thin Docker container for multiple VPN providers, written in Go, and using OpenVPN or Wireguard, DNS over TLS, with a few proxy servers built-in.
https://hub.docker.com/r/qmcgaw/gluetun
MIT License
7.42k stars 350 forks source link

Bug: cannot add route for interface: cannot add route {Ifindex: 9 Dst: ::/0 Src: <nil> Gw: <nil> Flags: [] Table: 51820 Realm: 0}: permission denied #998

Closed Th0masDB closed 2 years ago

Th0masDB commented 2 years ago

Is this urgent?

No

Host OS

Unraid 6.10.0

CPU arch

x86_64

VPN service provider

Custom

What are you using to run the container

Other

What is the version of Gluetun

Running version v3.29.0 built on 2022-05-11T23:16:02.058Z (commit e32d251)

What's the problem 🤔

The conainer is unhealthy because: cannot add route for interface: cannot add route {Ifindex: 9 Dst: ::/0 Src: <nil> Gw: <nil> Flags: [] Table: 51820 Realm: 0}: permission denied

Share your logs

├── VPN settings:
|   ├── VPN provider settings:
|   |   ├── Name: custom
|   |   └── Server selection settings:
|   |       ├── VPN type: wireguard
|   |       └── Wireguard selection settings:
|   |           ├── Endpoint IP address: 190.2.146.180
|   |           ├── Endpoint port: 51820
|   |           └── Server public key: EbxfUNJudEt6J4xL0kHH57eQM+P+OvypYxG4rpzE8iw=
|   └── Wireguard settings:
|       ├── Private key: wO...G0=
|       ├── Interface addresses:
|       |   └── 10.2.0.2/32
|       └── Network interface: wg1
├── DNS settings:
|   ├── DNS server address to use: 127.0.0.1
|   ├── Keep existing nameserver(s): yes
|   └── DNS over TLS settings:
|       ├── Enabled: yes
|       ├── Update period: every 24h0m0s
|       ├── Unbound settings:
|       |   ├── Authoritative servers:
|       |   |   └── quad9
|       |   ├── Caching: yes
|       |   ├── IPv6: yes
|       |   ├── Verbosity level: 1
|       |   ├── Verbosity details level: 0
|       |   ├── Validation log level: 0
|       |   ├── System user: root
|       |   └── Allowed networks:
|       |       ├── 0.0.0.0/0
|       |       └── ::/0
|       └── DNS filtering settings:
|           ├── Block malicious: yes
|           ├── Block ads: no
|           ├── Block surveillance: yes
|           └── Blocked IP networks:
|               ├── 127.0.0.1/8
|               ├── 10.0.0.0/8
|               ├── 172.16.0.0/12
|               ├── 192.168.0.0/16
|               ├── 169.254.0.0/16
|               ├── ::1/128
|               ├── fc00::/7
|               ├── fe80::/10
|               ├── ::ffff:7f00:1/104
|               ├── ::ffff:a00:0/104
|               ├── ::ffff:a9fe:0/112
|               ├── ::ffff:ac10:0/108
|               └── ::ffff:c0a8:0/112
├── Firewall settings:
|   ├── Enabled: yes
|   ├── Debug mode: on
|   └── Input ports:
|       └── 8191
├── Log settings:
|   └── Log level: INFO
├── Health settings:
|   ├── Server listening address: 127.0.0.1:9999
|   ├── Target address: github.com
|   └── VPN wait durations:
|       ├── Initial duration: 6s
|       └── Additional duration: 5s
├── Shadowsocks server settings:
|   ├── Enabled: yes
|   ├── Listening address: :8388
|   ├── Cipher: chacha20-ietf-poly1305
|   ├── Password: [set]
|   └── Log addresses: no
├── HTTP proxy settings:
|   ├── Enabled: yes
|   ├── Listening address: :8888
|   ├── User: admin
|   ├── Password: [set]
|   ├── Stealth mode: no
|   └── Log: no
├── Control server settings:
|   ├── Listening address: :8000
|   └── Logging: yes
├── OS Alpine settings:
|   ├── Process UID: 1000
|   ├── Process GID: 1000
|   └── Timezone: Europe/Amsterdam
├── Public IP settings:
|   ├── Fetching: every 12h0m0s
|   └── IP file path: /gluetun/ip
└── Version settings:
    └── Enabled: yes
2022-05-21T12:46:50+02:00 INFO [routing] default route found: interface eth0, gateway 172.17.0.1 and assigned IP 172.17.0.2
2022-05-21T12:46:50+02:00 INFO [routing] default route found: interface eth0, gateway fd17::1 and assigned IP 172.17.0.2
2022-05-21T12:46:50+02:00 DEBUG [routing] ip rule add from 172.17.0.2/32 lookup 200 pref 100
2022-05-21T12:46:50+02:00 DEBUG [routing] ip rule add from 172.17.0.2/32 lookup 200 pref 100
2022-05-21T12:46:50+02:00 INFO [routing] adding route for 0.0.0.0/0
2022-05-21T12:46:50+02:00 DEBUG [routing] ip route replace 0.0.0.0/0 via 172.17.0.1 dev eth0 table 200
2022-05-21T12:46:50+02:00 INFO [routing] adding route for ::/0
2022-05-21T12:46:50+02:00 DEBUG [routing] ip route replace ::/0 via fd17::1 dev eth0 table 200
2022-05-21T12:46:50+02:00 INFO [firewall] setting allowed subnets...
2022-05-21T12:46:50+02:00 INFO [routing] default route found: interface eth0, gateway 172.17.0.1 and assigned IP 172.17.0.2
2022-05-21T12:46:50+02:00 INFO [routing] default route found: interface eth0, gateway fd17::1 and assigned IP 172.17.0.2
2022-05-21T12:46:50+02:00 INFO TUN device is not available: open /dev/net/tun: no such file or directory; creating it...
2022-05-21T12:46:50+02:00 INFO [firewall] setting allowed input port 8191 through interface eth0...
2022-05-21T12:46:50+02:00 DEBUG [firewall] iptables --append INPUT -i eth0 -p tcp --dport 8191 -j ACCEPT
2022-05-21T12:46:50+02:00 DEBUG [firewall] ip6tables --append INPUT -i eth0 -p tcp --dport 8191 -j ACCEPT
2022-05-21T12:46:50+02:00 DEBUG [firewall] iptables --append INPUT -i eth0 -p udp --dport 8191 -j ACCEPT
2022-05-21T12:46:50+02:00 DEBUG [firewall] ip6tables --append INPUT -i eth0 -p udp --dport 8191 -j ACCEPT
2022-05-21T12:46:50+02:00 INFO [firewall] setting allowed input port 8191 through interface eth0...
2022-05-21T12:46:50+02:00 DEBUG [firewall] iptables --append INPUT -i eth0 -p tcp --dport 8191 -j ACCEPT
2022-05-21T12:46:50+02:00 DEBUG [firewall] ip6tables --append INPUT -i eth0 -p tcp --dport 8191 -j ACCEPT
2022-05-21T12:46:50+02:00 DEBUG [firewall] iptables --append INPUT -i eth0 -p udp --dport 8191 -j ACCEPT
2022-05-21T12:46:50+02:00 DEBUG [firewall] ip6tables --append INPUT -i eth0 -p udp --dport 8191 -j ACCEPT
2022-05-21T12:46:50+02:00 INFO [pprof] http server listening on [::]:6060
2022-05-21T12:46:50+02:00 INFO [http server] http server listening on [::]:8000
2022-05-21T12:46:50+02:00 INFO [dns over tls] using plaintext DNS at address 9.9.9.9
2022-05-21T12:46:50+02:00 INFO [http proxy] listening on :8888
2022-05-21T12:46:50+02:00 INFO [healthcheck] listening on 127.0.0.1:9999
2022-05-21T12:46:50+02:00 INFO [firewall] allowing VPN connection...
2022-05-21T12:46:50+02:00 DEBUG [firewall] iptables --append OUTPUT -d 190.2.146.180 -o eth0 -p udp -m udp --dport 51820 -j ACCEPT
2022-05-21T12:46:50+02:00 DEBUG [firewall] iptables --append OUTPUT -d 190.2.146.180 -o eth0 -p udp -m udp --dport 51820 -j ACCEPT
2022-05-21T12:46:50+02:00 DEBUG [firewall] iptables --append OUTPUT -o wg1 -j ACCEPT
2022-05-21T12:46:50+02:00 DEBUG [firewall] ip6tables --append OUTPUT -o wg1 -j ACCEPT
2022-05-21T12:46:50+02:00 INFO [shadowsocks] listening TCP on :8388
2022-05-21T12:46:50+02:00 INFO [shadowsocks] listening UDP on :8388
2022-05-21T12:46:50+02:00 INFO [wireguard] Using available kernelspace implementation
2022-05-21T12:46:50+02:00 INFO [wireguard] Connecting to 190.2.146.180:51820
2022-05-21T12:46:50+02:00 ERROR [vpn] cannot add route for interface: cannot add route {Ifindex: 8 Dst: ::/0 Src: <nil> Gw: <nil> Flags: [] Table: 51820 Realm: 0}: permission denied
2022-05-21T12:46:50+02:00 INFO [vpn] retrying in 15s
2022-05-21T12:46:56+02:00 INFO [healthcheck] program has been unhealthy for 6s: restarting VPN
2022-05-21T12:47:05+02:00 INFO [firewall] allowing VPN connection...
2022-05-21T12:47:05+02:00 INFO [wireguard] Using available kernelspace implementation
2022-05-21T12:47:05+02:00 INFO [wireguard] Connecting to 190.2.146.180:51820
2022-05-21T12:47:05+02:00 ERROR [vpn] cannot add route for interface: cannot add route {Ifindex: 9 Dst: ::/0 Src: <nil> Gw: <nil> Flags: [] Table: 51820 Realm: 0}: permission denied
2022-05-21T12:47:05+02:00 INFO [vpn] retrying in 30s
2022-05-21T12:47:07+02:00 INFO [healthcheck] program has been unhealthy for 11s: restarting VPN
2022-05-21T12:47:23+02:00 INFO [healthcheck] program has been unhealthy for 16s: restarting VPN
2022-05-21T12:47:35+02:00 INFO [firewall] allowing VPN connection...
2022-05-21T12:47:35+02:00 INFO [wireguard] Using available kernelspace implementation
2022-05-21T12:47:35+02:00 INFO [wireguard] Connecting to 190.2.146.180:51820
2022-05-21T12:47:35+02:00 ERROR [vpn] cannot add route for interface: cannot add route {Ifindex: 10 Dst: ::/0 Src: <nil> Gw: <nil> Flags: [] Table: 51820 Realm: 0}: permission denied
2022-05-21T12:47:35+02:00 INFO [vpn] retrying in 1m0s
2022-05-21T12:47:44+02:00 INFO [healthcheck] program has been unhealthy for 21s: restarting VPN

Share your configuration

No response

EDIT

I did check if my password, username etc were correct. I have ProtonVPN.

qdm12 commented 2 years ago

Hi there, that's a strange issue. It's due to your OS/docker setup/kernel not allowing to add a route.

Can you try

docker run -it --rm --cap-add=NET_ADMIN alpine:3.15 ip route

If it works, try

docker run -it --rm  --cap-add=NET_ADMIN alpine:3.15 ip route add default  0.0.0.0/0 dev eth0

What error do you get if any?

Th0masDB commented 2 years ago

I get this error:

image

qdm12 commented 2 years ago

Sorry the second one was a bad command from me. So the first one works that's good. Also sorry for the delay answering.

Anyway, I digged in the code, this comes from the Wireguard code. I have some code to detect if the container supports IPv6, and it looks like it does, so it tries to add the route for IPv6 destination ::/0 and that's where it fails.

I have pushed 7fd45cf17f53e007022d77ad5827e1c1d09c39e2 for the latest image, you can enable debug logs with LOG_LEVEL=debug (and also don't forget to docker pull qmcgaw/gluetun). What we're looking for is the logs starting from Checking for IPv6 support.... Then it should debug log all the interfaces with their IPv6 routes. Please share what you get, so I can fix the checking for IPv6 support. Thanks!

darovic commented 2 years ago

I believe I am experiencing the same problem after the most recent unRAID OS update. My log is attached (it's the raw file from the server, so it is in json format - I have redacted the client private key from the output, but it is correct). log-json.log

qdm12 commented 2 years ago

@darovic it looks like both lo (local loopback) and eth0 have ipv6 routes, so I'm not sure why adding ipv6 routes is denied permission...

How about

docker run --rm  --cap-add=NET_ADMIN alpine:3.15 ip -6 addr

what's the result?

If nothing is obvious, I'll just make it log out the error and continue execution.

darovic commented 2 years ago 
root@Imhotep:~# docker run --rm --cap-add=NET_ADMIN alpine:3.15 ip -6 addr
Unable to find image 'alpine:3.15' locally
3.15: Pulling from library/alpine
df9b9388f04a: Already exists
Digest: sha256:4edbd2beb5f78b1014028f4fbb99f3237d9561100b6881aabbf5acce2c4f9454
Status: Downloaded newer image for alpine:3.15
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
40: eth0@if41: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 state UP
    inet6 fd17::242:ac11:2/64 scope global flags 02
       valid_lft forever preferred_lft forever
    inet6 fe80::42:acff:fe11:2/64 scope link tentative
       valid_lft forever preferred_lft forever
qdm12 commented 2 years ago

eb18eaf0a9f953109b1079ab4c957844ee0d395d now logs out an error line linking back to this issue, but execution continues. I'm not 100% sure IPv6 would or would not leak out of Wireguard, so feel free to test it with the latest image. If you ever find a fix please report it here obviously. I'll close the issue for now since there is an ugly-but-working work-around in place.