search

mullvad / mullvadvpn-app

The Mullvad VPN client app for desktop and mobile
https://mullvad.net/
GNU General Public License v3.0
4.88k stars 335 forks source link

Ubuntu 20.04 2021.1 wont connect #2457

Closed macducknut closed 3 years ago

macducknut commented 3 years ago

The app starts cant connect to any servers Reinstalled the app and purged all settings tested again signing in did not work. reverted back to 2020.7 everything worked again so problem fixed for now but does anyone have an idea what the problem can be ?

faern commented 3 years ago

Please contact our support at support@mullvad.net. They can help you better than we can here.

Artturin commented 3 years ago

fyi i have the same issue and downgrading fixed it. im on NixOS

allenwalker3 commented 3 years ago

Same here. Ubuntu 20.04 AMD64; 2021.2 and 2021.1 both won't connect. Wireguard UDP. Just says "connecting". See:

Feb 14 11:32:11 firewall mullvad-daemon[2835]: [talpid_core::tunnel::wireguard][#033[33mWARN#033[0m] Timeout while checking tunnel connection
Feb 14 11:32:11 firewall systemd-networkd[736]: wg-mullvad: Link DOWN
Feb 14 11:32:11 firewall systemd-networkd[736]: wg-mullvad: Lost carrier
Feb 14 11:32:11 firewall networkd-dispatcher[10553]: #033[0;1;31mInterface "wg-mullvad" not found.#033[0m
Feb 14 11:32:11 firewall systemd[1]: networkd-dispatcher.service: Got notification message from PID 10553, but reception only permitted for main PID 762
Feb 14 11:32:11 firewall networkd-dispatcher[762]: ERROR:Failed to get interface "wg-mullvad" status: Command '['/usr/bin/networkctl', 'status', '--no-pager', '--no-legend', '--', 'wg-mullvad']' returned non-zero exit status 1.
Feb 14 11:32:11 firewall mullvad-daemon[2835]: [talpid_core::tunnel_state_machine::connecting_state][#033[34mDEBUG#033[0m] WireGuard tunnel timed out
Feb 14 11:32:11 firewall mullvad-daemon[2835]: [talpid_core::tunnel_state_machine::connecting_state][#033[34mDEBUG#033[0m] Tunnel monitor exited with block reason: None
Feb 14 11:32:11 firewall mullvad-daemon[2835]: [talpid_core::tunnel_state_machine::connecting_state][#033[32mINFO#033[0m] Tunnel closed. Reconnecting, attempt 2.
Feb 14 11:32:11 firewall mullvad-daemon[2835]: [talpid_core::routing::imp::imp][#033[34mDEBUG#033[0m] Clearing routes
Feb 14 11:32:11 firewall mullvad-daemon[2835]: [talpid_core::routing::imp::imp][#033[31mERROR#033[0m] Failed to remove route - ::/0 via dev wg-mullvad table 1836018789 - Netlink error
Feb 14 11:32:11 firewall mullvad-daemon[2835]: [mullvad_daemon::relays][#033[34mDEBUG#033[0m] Selecting among 7 relays with combined weight 35
Feb 14 11:32:11 firewall mullvad-daemon[2835]: [mullvad_daemon::relays][#033[32mINFO#033[0m] Selected relay us46-wireguard at 198.54.128.42
Feb 14 11:32:11 firewall mullvad-daemon[2835]: [mullvad_daemon::relays][#033[34mDEBUG#033[0m] Relay matched on highest preference for retry attempt 2
Feb 14 11:32:11 firewall mullvad-daemon[2835]: [talpid_core::firewall][#033[32mINFO#033[0m] Applying firewall policy: Connecting to 198.54.128.42:53 over UDP with gateways 10.64.0.1,fc00:bbbb:bbbb:bb01::1, Allowing LAN
Feb 14 11:32:11 firewall systemd-udevd[10554]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Feb 14 11:32:11 firewall systemd-udevd[10554]: Using default interface naming scheme 'v245'.
Feb 14 11:32:11 firewall networkd-dispatcher[762]: WARNING:Unknown index 43 seen, reloading interface list
Feb 14 11:32:11 firewall mullvad-daemon[2835]: [talpid_core::tunnel::wireguard][#033[34mDEBUG#033[0m] Using kernel WireGuard implementation
Feb 14 11:32:11 firewall mullvad-daemon[2835]: [talpid_core::routing::imp::imp][#033[34mDEBUG#033[0m] Adding routes: {RequiredRoute { prefix: V4(Ipv4Network { addr: 10.64.0.1, prefix: 32 }), node: RealNode(Node { ip: None, device: Some("wg-mullvad") }), table_id: 254 }, RequiredRoute { prefix: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), node: RealNode(Node { ip: None, device: Some("wg-mullvad") }), table_id: 1836018789 }, RequiredRoute { prefix: V6(Ipv6Network { addr: ::, prefix: 0 }), node: RealNode(Node { ip: None, device: Some("wg-mullvad") }), table_id: 1836018789 }, RequiredRoute { prefix: V6(Ipv6Network { addr: fc00:bbbb:bbbb:bb01::1, prefix: 128 }), node: RealNode(Node { ip: None, device: Some("wg-mullvad") }), table_id: 254 }}
Feb 14 11:32:11 firewall mullvad-daemon[2835]: [mullvad_daemon][#033[34mDEBUG#033[0m] New tunnel state: Connecting { endpoint: TunnelEndpoint { endpoint: Endpoint { address: 198.54.128.42:53, protocol: Udp }, tunnel_type: Wireguard, proxy: None }, location: Some(GeoIpLocation { ipv4: None, ipv6: None, country: "USA", city: Some("Denver, CO"), latitude: 39.7392358, longitude: -104.990251, mullvad_exit_ip: true, hostname: Some("us46-wireguard"), bridge_hostname: None }) }
macducknut commented 3 years ago

@auswalk did you find a solution to the problem yet ?

allenwalker3 commented 3 years ago

@macducknut No I gave up on 2021.2 . I went back to 2020.7 until a new version comes out I suppose.

macducknut commented 3 years ago

@auswalk same here tested 2021.2 what resolver do you use for ip and dns?

allenwalker3 commented 3 years ago

@macducknut I use adguard home actually along with the standard /etc/resolv.conf stuff. I tried setting the custom-dns prop in 2021.x but no dice. I have require-vpn off also just in case that is causing some blocking.

macducknut commented 3 years ago

Ok i mean are you using the normal network-manager as the resolver or like dhclient not so tech savy this can be called something else then a resolver

allenwalker3 commented 3 years ago

I use networkd in my netplan.io yaml config. And you?

macducknut commented 3 years ago

Dhclient. same problem then so it's not that . Hmm can it be something wrong with routing ?

allenwalker3 commented 3 years ago

@faern Why was this issue closed? Is there a solution?

faern commented 3 years ago

Could you please send a problem report from within the app? That way we get the full daemon logs. I don't think the above snippet contains what we need to debug this.

pinkisemils commented 3 years ago

@auswalk Would you mind sharing the netplan config files? We have never tested our app with netplan, and I'm curious if it's doing something that ends up being wildly incompatible with our routing setup. Although it most definitely shouldn't. What's the output of ip route and ip -6 route?

@macducknut what network configuration tool are you using? Are you using a plain dhclient? When is it invoked?

pinkisemils commented 3 years ago

@Artturin what's your nixOS networking config? I'm running nixOS myself, and I have been pretty much using master versions since forever. It very much feels like a common usecase that we haven't been testing or developing for. Granted, my networking config is rather pedestrian - 

networking.networkmanager.enable = true;
networking.iproute2.enable = true;
networking.firewall.enable = false;
pinkisemils commented 3 years ago

Also, have you guys checked if you have a valid WireGuard key? Or maybe you have a firewall installed?

macducknut commented 3 years ago

It says this on my netplan but om using dhclient

# Let NetworkManager manage all devices on this system network: version: 2 renderer: NetworkManager

Is there any privacy risk posting the ip route ?

And i Only use openvpn dont use wireguard

Im using dlclient changed it so network-manager is not the default choice allways liked dhclient more . It's the plain dhclient that comes with ubuntu 20.04

I have a firewall activated it's the firewall that comes with ubuntu no changed rules only activated

pinkisemils commented 3 years ago

You'd be giving up the routes that are set on your system - if you're using some specific small subset from a 172.16.0.0/16 or a 10.0.0.0/8, maybe. But if you are unsure, just err on the side of safety and don't post them. As long as that is the only thing you've configured in netplan and you're using dhclient, I'll see if I can replicate the issue.
Is there a specific reason you're specifying NetworkManager and then using dhclient?

macducknut commented 3 years ago

Well for me sometimes network-manager shows the original ip adress but its like only for a second in the firewall in the report window then it disappears it kind of bugs me out it's more of a feeling irritation from my side that i dont know why its showing it.. so I switched to dhclient and the problem dissapeared for me

macducknut commented 3 years ago

Also I have the ipv6 adress disabled from sysctl and from gnome networkmanager

macducknut commented 3 years ago

Well thats weird tried mullvad 2021.2 again and now it works but the Gnome network indikator is not switching icons when moving over to a another network

allenwalker3 commented 3 years ago

@auswalk Would you mind sharing the netplan config files? We have never tested our app with netplan, and I'm curious if it's doing something that ends up being wildly incompatible with our routing setup. Although it most definitely shouldn't. What's the output of ip route and ip -6 route?

@macducknut what network configuration tool are you using? Are you using a plain dhclient? When is it invoked?

$ cat /etc/netplan/00-installer-config.yaml
# This is the network config written by 'subiquity'
network:
  ethernets:
    enp1s0:
      dhcp4: true
      dhcp4-overrides:
        use-dns: false
    enp2s0:
      dhcp4: false
      addresses:
        - 192.168.2.1/24
        - "fd00::1/64"
      nameservers:
        addresses: [ 127.0.0.1 ]
        search: [ "lan" ]
  version: 2

Mullvad-2020.7

$ ip route
0.0.0.0/1 dev wg-mullvad proto static scope link
default via 192.168.99.1 dev enp1s0 proto dhcp src 192.168.99.175 metric 100
128.0.0.0/1 dev wg-mullvad proto static scope link
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
192.168.2.0/24 dev enp2s0 proto kernel scope link src 192.168.2.1
192.168.99.0/24 dev enp1s0 proto kernel scope link src 192.168.99.175
192.168.99.1 dev enp1s0 proto dhcp scope link src 192.168.99.175 metric 100
198.54.128.YZ via 192.168.99.1 dev enp1s0 proto static

$ ip -6 route
::1 dev lo proto kernel metric 256 pref medium
::/1 dev wg-mullvad proto static metric 1024 pref medium
fc00:bbbb:bbbb:bb01::x:xxxx dev wg-mullvad proto kernel metric 256 pref medium
fd00::/64 dev enp2s0 proto kernel metric 256 pref medium
fd00::/64 dev enp2s0 proto ra metric 1024 expires 86395sec pref medium
fe80::/64 dev enp2s0 proto kernel metric 256 pref medium
fe80::/64 dev enp1s0 proto kernel metric 256 pref medium
8000::/1 dev wg-mullvad proto static metric 1024 pref medium

Mullvad-2021.2

$ ip route
default via 192.168.99.1 dev enp1s0 proto dhcp src 192.168.99.175 metric 100
10.64.0.1 dev wg-mullvad proto static
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
192.168.2.0/24 dev enp2s0 proto kernel scope link src 192.168.2.1
192.168.99.0/24 dev enp1s0 proto kernel scope link src 192.168.99.175
192.168.99.1 dev enp1s0 proto dhcp scope link src 192.168.99.175 metric 100

$ ip -6 route
::1 dev lo proto kernel metric 256 pref medium
fc00:bbbb:bbbb:bb01::1 dev wg-mullvad proto static metric 1024 pref medium
fc00:bbbb:bbbb:bb01::x:xxxx dev wg-mullvad proto kernel metric 256 pref medium
fd00::/64 dev enp2s0 proto kernel metric 256 pref medium
fd00::/64 dev enp2s0 proto ra metric 1024 expires 86392sec pref medium
fe80::/64 dev enp2s0 proto kernel metric 256 pref medium
fe80::/64 dev enp1s0 proto kernel metric 256 pref medium

/var/log/syslog

Feb 27  firewall mullvad-daemon[712]: [talpid_core::tunnel::wireguard][#033[33mWARN#033[0m] Timeout while checking tunnel connection
Feb 27  firewall systemd-networkd[679]: wg-mullvad: Link DOWN
Feb 27  firewall systemd-networkd[679]: wg-mullvad: Lost carrier
Feb 27  firewall systemd-timesyncd[632]: Network configuration changed, trying to establish connection.

Feb 27  firewall mullvad-daemon[712]: [talpid_core::tunnel_state_machine::connecting_state][#033[34mDEBUG#033[0m] WireGuard tunnel timed out
Feb 27  firewall mullvad-daemon[712]: [talpid_core::tunnel_state_machine::connecting_state][#033[34mDEBUG#033[0m] Tunnel monitor exited with block reason: None
Feb 27  firewall mullvad-daemon[712]: [talpid_core::tunnel_state_machine::connecting_state][#033[34mDEBUG#033[0m] The tunnel disconnected unexpectedly
Feb 27  firewall mullvad-daemon[712]: [talpid_core::routing::imp::imp][#033[34mDEBUG#033[0m] Clearing routes
Feb 27  firewall mullvad-daemon[712]: [talpid_core::routing::imp::imp][#033[31mERROR#033[0m] Failed to remove route - ::/0 via dev wg-mullvad table 1836018789 - Netlink error
Feb 27  firewall mullvad-daemon[712]: [mullvad_daemon][#033[34mDEBUG#033[0m] New tunnel state: Disconnecting(Reconnect)
Feb 27  firewall mullvad-daemon[712]: [mullvad_daemon::relays][#033[34mDEBUG#033[0m] Selecting among 170 relays with combined weight 91258
Feb 27  firewall mullvad-daemon[712]: [mullvad_daemon::relays][#033[32mINFO#033[0m] Selected relay us54-wireguard at 89.45.90.15
Feb 27  firewall mullvad-daemon[712]: [mullvad_daemon::relays][#033[34mDEBUG#033[0m] Relay matched on highest preference for retry attempt 8
Feb 27  firewall mullvad-daemon[712]: [talpid_core::firewall][#033[32mINFO#033[0m] Applying firewall policy: Connecting to 89.45.90.15:12595 over UDP with gateways 10.64.0.1,fc00:bbbb:bbbb:bb01::1, Allowing LAN
Feb 27  firewall systemd-udevd[3592]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Feb 27  firewall networkd-dispatcher[716]: WARNING:Unknown index 14 seen, reloading interface list
Feb 27  firewall systemd-timesyncd[632]: Network configuration changed, trying to establish connection.

Feb 27  firewall systemd-udevd[3592]: Using default interface naming scheme 'v245'.
Feb 27  firewall systemd-timesyncd[632]: Network configuration changed, trying to establish connection.
Feb 27  firewall mullvad-daemon[712]: [talpid_core::tunnel::wireguard][#033[34mDEBUG#033[0m] Using kernel WireGuard implementation
Feb 27  firewall mullvad-daemon[712]: [talpid_core::routing::imp::imp][#033[34mDEBUG#033[0m] Adding routes: {RequiredRoute { prefix: V6(Ipv6Network { addr: fc00:bbbb:bbbb:bb01::1, prefix: 128 }), node: RealNode(Node { ip: None, device: Some("wg-mullvad") }), table_id: 254 }, RequiredRoute { prefix: V6(Ipv6Network { addr: ::, prefix: 0 }), node: RealNode(Node { ip: None, device: Some("wg-mullvad") }), table_id: 1836018789 }, RequiredRoute { prefix: V4(Ipv4Network { addr: 10.64.0.1, prefix: 32 }), node: RealNode(Node { ip: None, device: Some("wg-mullvad") }), table_id: 254 }, RequiredRoute { prefix: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), node: RealNode(Node { ip: None, device: Some("wg-mullvad") }), table_id: 1836018789 }}
Feb 27  firewall mullvad-daemon[712]: [mullvad_daemon][#033[34mDEBUG#033[0m] New tunnel state: Connecting { endpoint: TunnelEndpoint { endpoint: Endpoint { address: 89.45.90.15:12595, protocol: Udp }, tunnel_type: Wireguard, proxy: None }, location: Some(GeoIpLocation { ipv4: None, ipv6: None, country: "USA", city: Some("Los Angeles, CA"), latitude: 34.052235, longitude: -118.243683, mullvad_exit_ip: true, hostname: Some("us54-wireguard"), bridge_hostname: None }) }
Feb 27  firewall systemd-timesyncd[632]: Network configuration changed, trying to establish connection.
macducknut commented 3 years ago

I changed some adress fields with xx.xx.xx.xxx becuse I dont really know what's safe to show or not

ip route default via xx.xx.xx.x dev enp3s0 proto dhcp metric 100 xx.x.x.x/xx dev tun0 proto kernel scope link src 10.6.0.2 xx.xx.xx.x/xx dev enp3s0 proto kernel scope link src xx.xx.xx.xxx metric 100 xxx.xxx.x.x/xx dev enp3s0 scope link metric 1000

ip -6 route

is blank

What do you mean by network tool ? I'm using plain dhclient

What do you mean by invoked ? Do you mean when dhclient is starting ? At boot

This is my netplan

cat /etc/netplan/01-network-manager-all.yaml

Let NetworkManager manage all devices on this system

network: version: 2 renderer: NetworkManager

pinkisemils commented 3 years ago

With the netplan's above, (using one or multiple interfaces, using NetworkManager and networkd), I can't reproduce the issue. Could there be something else that's similar across these configs?

allenwalker3 commented 3 years ago

@pinkisemils

With the netplan's above, (using one or multiple interfaces, using NetworkManager and networkd), I can't reproduce the issue. Could there be something else that's similar across these configs?

If you're referring to my post, it's a clean Ubuntu 20.04-2 install on AMD64 hardware. 2020.7 works no problems. 2021.2 just says "connecting" and then times out.

pinkisemils commented 3 years ago

Just installed 20.04-2 Desktop (thus, with NM) on KVM/Qemu and it just works. So there must be something else. ~I'll try and see if it's the upgrade from 2020.7 to 2021.1/2 that breaks it, but that most definitely should not happen.~ Upgrading from 2020.7 to 2021.2 also doesn't magically reproduce the issue.

Are you guys using any kind of a firewall by any chance? Truly grasping at straws here. I'll try and see if there's any significant difference between tunnel configs that are applied between 2020.7 and 2021.2. If you want, you can dump the firewall rules via nftables - 

sudo apt install nftables
nft list ruleset

I expect to only see mullvad related rulesets there.

allenwalker3 commented 3 years ago

Ubuntu-desktop has network-manager installed. I'll try installing that and using "renderer: NetworkManager" in my netplan.yml

pinkisemils commented 3 years ago

There must be a bug somewhere, because a lot of users are reporting this and a lot of users are reporting that an older version continues to work. But it's incredibly difficult to fix something that isn't broken locally.

allenwalker3 commented 3 years ago

@pinkisemils No dice with network-manager installed, netplan renderer: NetworkManager, mullvad-vpn 2021.2; also I run ufw firewall but disabled it for this test.

/var/log/syslog

Mar  1  firewall mullvad-daemon[2921]: [talpid_core::tunnel::wireguard][#033[33mWARN#033[0m] Timeout while checking tunnel connection
Mar  1  firewall systemd-networkd[684]: wg-mullvad: Link DOWN
Mar  1  firewall systemd-networkd[684]: wg-mullvad: Lost carrier
Mar  1  firewall systemd-timesyncd[639]: Network configuration changed, trying to establish connection.
Mar  1  firewall systemd-timesyncd[639]: Network configuration changed, trying to establish connection.
Mar  1  firewall NetworkManager[698]: <info>  [1614635569.1687] device (wg-mullvad): state change: activated -> unmanaged (reason 'unmanaged', sys-iface-state: 'removed')
Mar  1  firewall dbus-daemon[697]: [system] Activating via systemd: service name='org.freedesktop.nm_dispatcher' unit='dbus-org.freedesktop.nm-dispatcher.service' requested by ':1.8' (uid=0 pid=698 comm="/usr/sbin/NetworkManager --no-daemon " label="unconfined")
Mar  1  firewall systemd[1]: Starting Network Manager Script Dispatcher Service...
Mar  1  firewall dbus-daemon[697]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher'
Mar  1  firewall systemd[1]: Started Network Manager Script Dispatcher Service.
Mar  1  firewall mullvad-daemon[2921]: [talpid_core::tunnel_state_machine::connecting_state][#033[34mDEBUG#033[0m] WireGuard tunnel timed out
Mar  1  firewall mullvad-daemon[2921]: [talpid_core::tunnel_state_machine::connecting_state][#033[34mDEBUG#033[0m] Tunnel monitor exited with block reason: None
Mar  1  firewall mullvad-daemon[2921]: [talpid_core::tunnel_state_machine::connecting_state][#033[32mINFO#033[0m] Tunnel closed. Reconnecting, attempt 1.
Mar  1  firewall mullvad-daemon[2921]: [talpid_core::routing::imp::imp][#033[34mDEBUG#033[0m] Clearing routes
Mar  1  firewall mullvad-daemon[2921]: [talpid_core::routing::imp::imp][#033[31mERROR#033[0m] Failed to remove route - ::/0 via dev wg-mullvad table 1836018789 - Netlink error
Mar  1  firewall mullvad-daemon[2921]: [mullvad_daemon::relays][#033[34mDEBUG#033[0m] Selecting among 7 relays with combined weight 35
Mar  1  firewall mullvad-daemon[2921]: [mullvad_daemon::relays][#033[32mINFO#033[0m] Selected relay us10-wireguard at 198.54.128.82
Mar  1  firewall mullvad-daemon[2921]: [mullvad_daemon::relays][#033[34mDEBUG#033[0m] Relay matched on highest preference for retry attempt 1
Mar  1  firewall mullvad-daemon[2921]: [talpid_core::firewall][#033[32mINFO#033[0mr] Applying firewall policy: Connecting to 198.54.128.82:45229 over UDP with gateways 10.64.0.1,fc00:bbbb:bbbb:bb01::1, Allowing LAN
Mar  1  firewall NetworkManager[698]: <info>  [1614635569.2777] manager: (wg-mullvad): new WireGuard device (/org/freedesktop/NetworkManager/Devices/12)
Mar  1  firewall systemd-timesyncd[639]: Network configuration changed, trying to establish connection.
Mar  1  firewall networkd-dispatcher[707]: WARNING:Unknown index 12 seen, reloading interface list
Mar  1  firewall systemd-udevd[4830]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Mar  1  firewall systemd-udevd[4830]: Using default interface naming scheme 'v245'.
Mar  1  firewall systemd-timesyncd[639]: Network configuration changed, trying to establish connection.
Mar  1  firewall mullvad-daemon[2921]: [talpid_core::tunnel::wireguard][#033[34mDEBUG#033[0m] Using kernel WireGuard implementation
Mar  1  firewall mullvad-daemon[2921]: [talpid_core::routing::imp::imp][#033[34mDEBUG#033[0m] Adding routes: {RequiredRoute { prefix: V4(Ipv4Network { addr: 10.64.0.1, prefix: 32 }), node: RealNode(Node { ip: None, device: Some("wg-mullvad") }), table_id: 254 }, RequiredRoute { prefix: V6(Ipv6Network { addr: fc00:bbbb:bbbb:bb01::1, prefix: 128 }), node: RealNode(Node { ip: None, device: Some("wg-mullvad") }), table_id: 254 }, RequiredRoute { prefix: V6(Ipv6Network { addr: ::, prefix: 0 }), node: RealNode(Node { ip: None, device: Some("wg-mullvad") }), table_id: 1836018789 }, RequiredRoute { prefix: V4(Ipv4Network { addr: 0.0.0.0, prefix: 0 }), node: RealNode(Node { ip: None, device: Some("wg-mullvad") }), table_id: 1836018789 }}
Mar  1  firewall mullvad-daemon[2921]: [mullvad_daemon][#033[34mDEBUG#033[0m] New tunnel state: Connecting { endpoint: TunnelEndpoint { endpoint: Endpoint { address: 198.54.128.82:45229, protocol: Udp }, tunnel_type: Wireguard, proxy: None }, location: Some(GeoIpLocation { ipv4: None, ipv6: None, country: "USA", city: Some("Denver, CO"), latitude: 39.7392358, longitude: -104.990251, mullvad_exit_ip: true, hostname: Some("us10-wireguard"), bridge_hostname: None }) }
Mar  1  firewall NetworkManager[698]: <info>  [1614635569.3819] device (wg-mullvad): state change: unmanaged -> unavailable (reason 'connection-assumed', sys-iface-state: 'external')
Mar  1  firewall NetworkManager[698]: <info>  [1614635569.3906] device (wg-mullvad): state change: unavailable -> disconnected (reason 'connection-assumed', sys-iface-state: 'external')
Mar  1  firewall NetworkManager[698]: <info>  [1614635569.3977] device (wg-mullvad): Activation: starting connection 'wg-mullvad' (77d46bde-ab09-4261-b27d-af5074c3c518)
Mar  1  firewall NetworkManager[698]: <info>  [1614635569.3981] device (wg-mullvad): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'external')
Mar  1  firewall systemd-timesyncd[639]: Network configuration changed, trying to establish connection.
Mar  1  firewall NetworkManager[698]: <info>  [1614635569.3996] device (wg-mullvad): state change: prepare -> config (reason 'none', sys-iface-state: 'external')
Mar  1  firewall NetworkManager[698]: <info>  [1614635569.4026] device (wg-mullvad): state change: config -> ip-config (reason 'none', sys-iface-state: 'external')
Mar  1  firewall NetworkManager[698]: <info>  [1614635569.4044] device (wg-mullvad): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'external')
Mar  1  firewall NetworkManager[698]: <info>  [1614635569.4116] device (wg-mullvad): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'external')
Mar  1  firewall NetworkManager[698]: <info>  [1614635569.4132] device (wg-mullvad): state change: secondaries -> activated (reason 'none', sys-iface-state: 'external')
Mar  1  firewall NetworkManager[698]: <info>  [1614635569.4187] device (wg-mullvad): Activation: successful, device activated.
allenwalker3 commented 3 years ago

Looks like talpid_core might be worth investigating as the culprit?

pinkisemils commented 3 years ago

The failure to remove routes is there only because NM has already removed the interface, which automatically removes the route. Maybe there are routing rules that we do not expect - ip rule and ip -6 rule would show them. But in a bog standard install of Ubuntu 20.04, this shouldn't be an issue.

pinkisemils commented 3 years ago

I've managed to reproduce the issue by enabling the firewall service on NixOS.

pinkisemils commented 3 years ago

Do any of you have enabled strict reverse path filtering at the kernel level? I'm interested in the output of sysctl -ar '\.rp_filter'.

allenwalker3 commented 3 years ago

This is Mullvad-2020.7 running and connected:

~$ sysctl -ar '\.rp_filter'
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.docker0.rp_filter = 1
net.ipv4.conf.enp1s0.rp_filter = 1
net.ipv4.conf.enp2s0.rp_filter = 1
net.ipv4.conf.lo.rp_filter = 0
net.ipv4.conf.usb0.rp_filter = 1
net.ipv4.conf.wg-mullvad.rp_filter = 1
pinkisemils commented 3 years ago

It'd be great if you could test the changes we made with a dev build. Mind you, this is a development build, we have not tested it at all and for all we know, it's wildly unsafe. So it's understandable if you'd prefer to wait until we make a beta release.

@auswalk I see you've enabled strict reverse path filtering. Have you personally and specifically enabled reverse path filtering? Is there a specific reason you've chosen to use the kernel built-in reverse path filtering over using an iptables or nftables based solution? For the above build to work, you can set src_valid_mark to 1 like so:

sysctl net.ipv4.conf.all.src_valid_mark=1

We are currently trying to decide if we should be changing people's kernel configuration on the fly - we really would prefer to not have to do this, as we expect most users set these things because they want to and know what they are doing.

The issue is that we've changed our client to use policy based routing to better support split tunneling. Strict reverse path filtering will fail in this case because it will not take our policy routing into account when deciding whether an incoming packet would be routed through the same interface it came in. But it will if src_valid_mark is enabled.

allenwalker3 commented 3 years ago

@pinkisemils I have enabled rp_filter

/etc/sysctl.conf

...
# Uncomment the next two lines to enable Spoof protection (reverse-path filter)
# Turn on Source Address Verification in all interfaces to
# prevent some spoofing attacks
net.ipv4.conf.default.rp_filter=1
net.ipv4.conf.all.rp_filter=1
...

Before I go down the dev build route, are you saying to comment these lines out and try mullvad-2021.2 again?

pinkisemils commented 3 years ago

Yes, but in the dev-build and an upcoming release, that shouldn't be necessary as long as src_valid_mark is enabled for the interfaces that will receive relay traffic.

allenwalker3 commented 3 years ago

It is now Connected! Mullvad-2021.2 ; Thanks!

allenwalker3 commented 3 years ago

@pinkisemils Also the custom-dns setting is working: if ipv6 is on and you're running something like a local AdGuardHome instance, you must set both the ip4 and ip6 addresses , i.e.:

$ mullvad custom-dns servers set 192.168.2.1 fd00::1

Note without the ipv6 address in the custom-dns variable, requests to fd00::1 port 53 are blocked, which is correct behavior.

macducknut commented 3 years ago

This is solved for me its a mystery tried to recreate it but no luck