jamesmcm / vopono

Run applications through VPN tunnels with temporary network namespaces
GNU General Public License v3.0
881 stars 46 forks source link

VPN connection fails with IPv6 endpoints #181

Open hashworks opened 2 years ago

hashworks commented 2 years ago

EDIT: The issue is the IPv6 endpoint – currently vopono has no support for those.


I'm trying to connect to Mullvad using the following custom config:

[Interface]
PrivateKey = redacted
Address = 10.65.23.123/32,fc00:bbbb:bbbb:bb01::2:abc/128
DNS = 100.64.0.23

[Peer]
PublicKey = redacted
AllowedIPs = 0.0.0.0/0,::0/0
Endpoint = [2a03:1b20:6:f011::a11f]:51820

Using this config with wg-quick on the host works just fine.

With vopono, I can't reach the Mullvad DNS IP or any public IP:

``` $ vopono -v exec -c Wireguard --custom /etc/wireguard/vopono/mullvad-de11.conf --no-killswitch bash 2022-08-18T16:47:53.309Z DEBUG vopono_core::util > Using config dir from $HOME config: /home/hashworks/.config 2022-08-18T16:47:53.314Z DEBUG vopono_core::util::pulseaudio > Setting PULSE_SERVER to /run/user/1000/pulse/native 2022-08-18T16:47:53.314Z INFO vopono_core::util > Calling sudo for elevated privileges, current user will be used as default user 2022-08-18T16:47:53.314Z DEBUG vopono_core::util > Args: ["vopono", "-v", "exec", "-c", "Wireguard", "--custom", "/etc/wireguard/vopono/mullvad-de11.conf", "--no-killswitch", "bash"] 2022-08-18T16:47:53.403Z DEBUG vopono_core::util > Using config dir from $HOME config: /home/hashworks/.config 2022-08-18T16:47:53.408Z DEBUG vopono_core::util::pulseaudio > Setting PULSE_SERVER to /run/user/1000/pulse/native 2022-08-18T16:47:53.408Z DEBUG vopono_core::util > Using config dir from $HOME config: /home/hashworks/.config 2022-08-18T16:47:53.409Z DEBUG vopono_core::util > Existing namespaces: [] 2022-08-18T16:47:53.409Z DEBUG vopono_core::util > Using config dir from $HOME config: /home/hashworks/.config 2022-08-18T16:47:53.409Z DEBUG vopono_core::util > Using config dir from $HOME config: /home/hashworks/.config 2022-08-18T16:47:53.410Z DEBUG vopono::exec > vopono config.toml: configuration property "firewall" not found 2022-08-18T16:47:53.410Z DEBUG vopono::exec > vopono config.toml: configuration property "custom_netns_name" not found 2022-08-18T16:47:53.410Z DEBUG vopono::exec > vopono config.toml: configuration property "open_hosts" not found 2022-08-18T16:47:53.410Z DEBUG vopono::exec > vopono config.toml: configuration property "allow_host_access" not found 2022-08-18T16:47:53.410Z DEBUG vopono::exec > vopono config.toml: configuration property "postup" not found 2022-08-18T16:47:53.410Z DEBUG vopono::exec > vopono config.toml: configuration property "predown" not found 2022-08-18T16:47:53.410Z DEBUG vopono::exec > vopono config.toml: configuration property "user" not found 2022-08-18T16:47:53.410Z DEBUG vopono::exec > vopono config.toml: configuration property "dns" not found 2022-08-18T16:47:53.410Z DEBUG vopono::exec > vopono config.toml: configuration property "interface" not found 2022-08-18T16:47:53.410Z DEBUG vopono_core::network::network_interface > ip addr 2022-08-18T16:47:53.411Z DEBUG vopono::exec > Interface: eno1 2022-08-18T16:47:53.412Z DEBUG vopono_core::util > Existing namespaces: [] 2022-08-18T16:47:53.412Z DEBUG vopono_core::util > ip netns add vopono_c_Xwt4g47Fkxa 2022-08-18T16:47:53.413Z INFO vopono_core::network::netns > Created new network namespace: vopono_c_Xwt4g47Fkxa 2022-08-18T16:47:53.414Z DEBUG vopono_core::util > Existing interfaces: 2022-08-18T16:47:53.415Z DEBUG vopono_core::util > Assigned IPs: [] 2022-08-18T16:47:53.415Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa ip addr add 127.0.0.1/8 dev lo 2022-08-18T16:47:53.417Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa ip link set lo up 2022-08-18T16:47:53.419Z DEBUG vopono_core::network::veth_pair > NetworkManager not detected running 2022-08-18T16:47:53.419Z DEBUG vopono_core::network::veth_pair > firewalld not detected running 2022-08-18T16:47:53.419Z DEBUG vopono_core::util > ip link add c_Xwt4g47Fkxa_d type veth peer name c_Xwt4g47Fkxa_s 2022-08-18T16:47:53.420Z DEBUG vopono_core::util > ip link set c_Xwt4g47Fkxa_d up 2022-08-18T16:47:53.422Z DEBUG vopono_core::util > ip link set c_Xwt4g47Fkxa_s netns vopono_c_Xwt4g47Fkxa up 2022-08-18T16:47:53.443Z DEBUG vopono_core::util > ip addr add 10.200.1.1/24 dev c_Xwt4g47Fkxa_d 2022-08-18T16:47:53.444Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa ip addr add 10.200.1.2/24 dev c_Xwt4g47Fkxa_s 2022-08-18T16:47:53.446Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa ip route add default via 10.200.1.1 dev c_Xwt4g47Fkxa_s 2022-08-18T16:47:53.447Z INFO vopono_core::network::netns > IP address of namespace as seen from host: 10.200.1.2 2022-08-18T16:47:53.447Z INFO vopono_core::network::netns > IP address of host as seen from namespace: 10.200.1.1 2022-08-18T16:47:53.447Z DEBUG vopono_core::util > iptables -t nat -A POSTROUTING -s 10.200.1.0/24 -o eno1 -j MASQUERADE 2022-08-18T16:47:53.449Z DEBUG vopono_core::util > iptables -I FORWARD -i c_Xwt4g47Fkxa_d -o eno1 -j ACCEPT 2022-08-18T16:47:53.449Z DEBUG vopono_core::util > iptables -I FORWARD -o c_Xwt4g47Fkxa_d -i eno1 -j ACCEPT 2022-08-18T16:47:53.450Z DEBUG vopono_core::util > sysctl -q net.ipv4.ip_forward=1 2022-08-18T16:47:53.451Z DEBUG vopono_core::network::wireguard > Deserializing: 100.64.0.23 to Vec 2022-08-18T16:47:53.451Z DEBUG vopono_core::network::wireguard > TOML config: WireguardConfig { interface: WireguardInterface { private_key: "redacted", address: [10.65.23.123/32, fc00:bbbb:bbbb:bb01::2:abc/128], dns: Some([100.64.0.23]) }, peer: WireguardPeer { public_key: "redacted", allowed_ips: [0.0.0.0/0, ::/0], endpoint: [2a03:1b20:6:f011::a11f]:51820 } } 2022-08-18T16:47:53.451Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa ip link add c_Xwt4g47Fkxa type wireguard 2022-08-18T16:47:53.453Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa wg setconf c_Xwt4g47Fkxa /tmp/vopono_nft.conf 2022-08-18T16:47:53.455Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa ip -4 address add 10.65.23.123/32 dev c_Xwt4g47Fkxa 2022-08-18T16:47:53.456Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa ip -6 address add fc00:bbbb:bbbb:bb01::2:abc/128 dev c_Xwt4g47Fkxa 2022-08-18T16:47:53.458Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa ip link set mtu 1420 up dev c_Xwt4g47Fkxa 2022-08-18T16:47:53.460Z DEBUG vopono_core::network::dns_config > Setting namespace vopono_c_Xwt4g47Fkxa DNS server to 100.64.0.23 2022-08-18T16:47:53.460Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa wg set c_Xwt4g47Fkxa fwmark 51820 2022-08-18T16:47:53.462Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa ip -4 route add 0.0.0.0/0 dev c_Xwt4g47Fkxa table 51820 2022-08-18T16:47:53.464Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa ip -4 rule add not fwmark 51820 table 51820 2022-08-18T16:47:53.465Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa ip -4 rule add table main suppress_prefixlength 0 2022-08-18T16:47:53.467Z DEBUG vopono_core::util > sysctl -q net.ipv4.conf.all.src_valid_mark=1 2022-08-18T16:47:53.468Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa ip -6 route add ::/0 dev c_Xwt4g47Fkxa table 51820 2022-08-18T16:47:53.470Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa ip -6 rule add not fwmark 51820 table 51820 2022-08-18T16:47:53.471Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa ip -6 rule add table main suppress_prefixlength 0 2022-08-18T16:47:53.473Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa iptables -t raw -A PREROUTING ! -i c_Xwt4g47Fkxa -d 10.65.23.123/32 -m addrtype ! --src-type LOCAL -j DROP 2022-08-18T16:47:53.475Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa ip6tables -t raw -A PREROUTING ! -i c_Xwt4g47Fkxa -d fc00:bbbb:bbbb:bb01::2:abc/128 -m addrtype ! --src-type LOCAL -j DROP 2022-08-18T16:47:53.476Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa iptables -t mangle -A POSTROUTING -p udp -j MARK --set-mark 51820 2022-08-18T16:47:53.478Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa iptables -t mangle -A PREROUTING -p udp -j CONNMARK --save-mark 2022-08-18T16:47:53.480Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa ip6tables -t mangle -A POSTROUTING -p udp -j MARK --set-mark 51820 2022-08-18T16:47:53.481Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa ip6tables -t mangle -A PREROUTING -p udp -j CONNMARK --save-mark 2022-08-18T16:47:53.483Z DEBUG vopono_core::util > Using config dir from $HOME config: /home/hashworks/.config 2022-08-18T16:47:53.483Z DEBUG vopono_core::network::netns > Writing lockfile: /home/hashworks/.config/vopono/locks/vopono_c_Xwt4g47Fkxa 2022-08-18T16:47:53.483Z DEBUG vopono_core::network::netns > Lockfile written: /home/hashworks/.config/vopono/locks/vopono_c_Xwt4g47Fkxa/22984 2022-08-18T16:47:53.483Z DEBUG vopono_core::util > Using config dir from $HOME config: /home/hashworks/.config 2022-08-18T16:47:53.493Z DEBUG vopono_core::network::netns > ip netns exec vopono_c_Xwt4g47Fkxa sudo -Eu hashworks bash 2022-08-18T16:47:53.494Z INFO vopono::exec > Application bash launched in network namespace vopono_c_Xwt4g47Fkxa with pid 23029 $ sudo ping 100.64.0.23 PING 100.64.0.23 (100.64.0.23) 56(84) bytes of data. ^C --- 100.64.0.23 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2024ms $ sudo ping 1.1.1.1 PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data. ^C --- 1.1.1.1 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2017ms $ traceroute 100.64.0.23 traceroute to 100.64.0.23 (100.64.0.23), 30 hops max, 60 byte packets 1 * * * [...] 30 * * * $ traceroute 1.1.1.1 traceroute to 1.1.1.1 (1.1.1.1), 30 hops max, 60 byte packets 1 * * * [...] 30 * * * $ ip addr 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: c_Xwt4g47Fkxa: mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000 link/none inet 10.65.23.123/32 scope global c_Xwt4g47Fkxa valid_lft forever preferred_lft forever inet6 fc00:bbbb:bbbb:bb01::2:abc/128 scope global valid_lft forever preferred_lft forever 14: c_Xwt4g47Fkxa_s@if15: mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether e2:51:46:fc:61:34 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 10.200.1.2/24 scope global c_Xwt4g47Fkxa_s valid_lft forever preferred_lft forever inet6 fe80::e051:46ff:fefc:6134/64 scope link valid_lft forever preferred_lft forever $ ip link 1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: c_Xwt4g47Fkxa: mtu 1420 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/none 14: c_Xwt4g47Fkxa_s@if15: mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 link/ether e2:51:46:fc:61:34 brd ff:ff:ff:ff:ff:ff link-netnsid 0 $ sudo iptables -L Chain INPUT (policy ACCEPT) target prot opt source destination Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination $ sudo ping 10.200.1.1 PING 10.200.1.1 (10.200.1.1) 56(84) bytes of data. 64 bytes from 10.200.1.1: icmp_seq=1 ttl=64 time=0.056 ms 64 bytes from 10.200.1.1: icmp_seq=2 ttl=64 time=0.027 ms ^C --- 10.200.1.1 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1001ms rtt min/avg/max/mdev = 0.027/0.041/0.056/0.014 ms ``` From the host: ``` $ ip addr 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eno1: mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether redacted brd ff:ff:ff:ff:ff:ff altname enp0s25 inet 192.168.144.123/24 brd 192.168.144.255 scope global eno1 valid_lft forever preferred_lft forever inet6 2a02:redacted/64 scope global temporary dynamic valid_lft 603733sec preferred_lft 85137sec inet6 2a02:redacted/64 scope global dynamic mngtmpaddr noprefixroute valid_lft 2591980sec preferred_lft 604780sec inet6 2a02:redacted/128 scope global valid_lft forever preferred_lft forever inet6 fe80::redacted/64 scope link valid_lft forever preferred_lft forever 15: c_Xwt4g47Fkxa_d@if14: mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether be:e2:6f:34:6d:58 brd ff:ff:ff:ff:ff:ff link-netns vopono_c_Xwt4g47Fkxa inet 10.200.1.1/24 scope global c_Xwt4g47Fkxa_d valid_lft forever preferred_lft forever inet6 fe80::bce2:6fff:fe34:6d58/64 scope link valid_lft forever preferred_lft forever $ ip link 1: lo: mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eno1: mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000 link/ether redacted brd ff:ff:ff:ff:ff:ff altname enp0s25 15: c_Xwt4g47Fkxa_d@if14: mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 link/ether be:e2:6f:34:6d:58 brd ff:ff:ff:ff:ff:ff link-netns vopono_c_Xwt4g47Fkxa $ ping 10.200.1.2 PING 10.200.1.2 (10.200.1.2) 56(84) bytes of data. 64 bytes from 10.200.1.2: icmp_seq=1 ttl=64 time=0.053 ms 64 bytes from 10.200.1.2: icmp_seq=2 ttl=64 time=0.031 ms ^C --- 10.200.1.2 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1007ms rtt min/avg/max/mdev = 0.031/0.042/0.053/0.011 ms $ sudo iptables -t nat -L Chain PREROUTING (policy ACCEPT) target prot opt source destination Chain INPUT (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination Chain POSTROUTING (policy ACCEPT) target prot opt source destination MASQUERADE all -- 10.200.1.0/24 anywhere ``` Any idea?
jamesmcm commented 2 years ago

Hmm I tested this locally with a US config from Mullvad and it works fine for me.

Can you check that ufw, firewalld are not running? i.e. anything that might interfere with the nftables rules?

Also I don't see that relay when I check (the IP address is different) - could you try re-running vopono sync ?

hashworks commented 2 years ago

Can you check that ufw, firewalld are not running? i.e. anything that might interfere with the nftables rules?

No firewall daemon is running. Also, I'm using iptables on that host, and for testing purposes all rules have been reset.

Also I don't see that relay when I check (the IP address is different) - could you try re-running vopono sync?

The relay exists, it is de11-wireguard: relay

Any vopono sync shouldn't be needed here since I'm using a custom config file.

hashworks commented 2 years ago

The issue seems to be the IPv6 Endpoint – when I use the IPv4 endpoint, vopono works just fine.

It seems like you didn't run into that issue yet because vopono sync always creates IPv4 configs.

jamesmcm commented 2 years ago

Yeah, this is reproducible just with the netns parts:

sudo ip netns add testnetns2
sudo ip netns exec testnetns2 ip addr add ::1/8 dev lo
sudo ip netns exec testnetns2 ip link set lo up
sudo ip link add testnetns2_d type veth peer name testnetns2_s
sudo ip link set testnetns2_d up
sudo ip link set testnetns2_s netns testnetns2 up
sudo ip addr add 10.200.3.1/24 dev testnetns2_d
sudo ip netns exec testnetns2 ip addr add 10.200.3.2/24 dev testnetns2_s
sudo ip netns exec testnetns2 ip route add default via 10.200.3.1 dev testnetns2_s
sudo ip netns exec testnetns2 ip -6 route add default via fe80::c418:42ff:fec9:62b6 dev testnetns2_s
sudo nft add table inet testnetns2_nat
sudo nft add chain inet testnetns2_nat postrouting { type nat hook postrouting priority 100 ; }
sudo nft add rule inet testnetns2_nat postrouting oifname enp3s0f3u1 ip saddr 10.200.3.0/24 counter masquerade
sudo nft add rule inet testnetns2_nat postrouting oifname enp3s0f3u1 ip6 saddr fe80::c418:42ff:fec9:62b6/24 counter masquerade
sudo nft add table inet testnetns2_bridge
sudo nft add chain inet testnetns2_bridge forward { type filter hook forward priority -10 ; }
sudo nft add rule inet testnetns2_bridge forward iifname testnetns2_d oifname enp3s0f3u1 counter accept
sudo nft add rule inet testnetns2_bridge forward oifname testnetns2_d iifname enp3s0f3u1 counter accept
sudo sysctl -q net.ipv4.ip_forward=1
sudo sysctl -w net.ipv6.conf.default.forwarding=1
sudo ip netns exec testnetns2 sysctl -w net.ipv6.conf.default.forwarding=1

Then inside the netns:

ping 8.8.8.8

works but:

ping 2001:4860:4860::8888

fails

I still can't get it to work atm, even just adjusting the NAT rules.

There's a good summary at https://stackoverflow.com/questions/36438102/ping-external-ipv6-address-from-a-network-namespace - it doesn't seem trivial to have it work for both IPv4 and IPv6 though.

Digitalone1 commented 4 months ago

I'm not an expert here, but this seems an issue with private IPv6 which are not routable.

Is there a way to use a global address retrieving the prefix delegated by the ISP?

Reading the question on stackoverflow, using the global addresses solve the issue.