canonical / multipass

Multipass orchestrates virtual Ubuntu instances
https://multipass.run
GNU General Public License v3.0
7.73k stars 641 forks source link

No internet access after VPN install (macOS). #1121

Open konart opened 4 years ago

konart commented 4 years ago

Clean multipass install: multipass instances work fine - you can ping any outer resources by their domain or IP.

As soon as https://www.checkpoint.com/products/remote-access-vpn/ is installed, even if the client is not connected (and actually not running) - my multipass instance does have internet access for the first 10-20 seconds, but then - dead silence. You can't access anything.

I've tried to figure it out somehow using these issues:

https://github.com/CanonicalLtd/multipass/issues/495 https://github.com/CanonicalLtd/multipass/issues/961

without any luck. Uninstalling VPN client doesn't help either.

Any ideas?

ifconfig:

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
    options=1203<RXCSUM,TXCSUM,TXSTATUS,SW_TIMESTAMP>
    inet 127.0.0.1 netmask 0xff000000
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
    nd6 options=201<PERFORMNUD,DAD>
gif0: flags=8010<POINTOPOINT,MULTICAST> mtu 1280
stf0: flags=0<> mtu 1280
en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    options=400<CHANNEL_IO>
    ether b8:e8:56:47:32:f8
    inet6 fe80::142b:dda6:3e72:28f4%en0 prefixlen 64 secured scopeid 0x4
    inet 172.30.4.80 netmask 0xffffff00 broadcast 172.30.4.255
    nd6 options=201<PERFORMNUD,DAD>
    media: autoselect
    status: active
en1: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
    options=460<TSO4,TSO6,CHANNEL_IO>
    ether 82:0f:03:7c:6d:80
    media: autoselect <full-duplex>
    status: inactive
en2: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
    options=460<TSO4,TSO6,CHANNEL_IO>
    ether 82:0f:03:7c:6d:81
    media: autoselect <full-duplex>
    status: inactive
bridge0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    options=63<RXCSUM,TXCSUM,TSO4,TSO6>
    ether 82:0f:03:7c:6d:80
    Configuration:
        id 0:0:0:0:0:0 priority 0 hellotime 0 fwddelay 0
        maxage 0 holdcnt 0 proto stp maxaddr 100 timeout 1200
        root id 0:0:0:0:0:0 priority 0 ifcost 0 port 0
        ipfilter disabled flags 0x2
    member: en1 flags=3<LEARNING,DISCOVER>
            ifmaxaddr 0 port 5 priority 0 path cost 0
    member: en2 flags=3<LEARNING,DISCOVER>
            ifmaxaddr 0 port 6 priority 0 path cost 0
    nd6 options=201<PERFORMNUD,DAD>
    media: <unknown type>
    status: inactive
p2p0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 2304
    options=400<CHANNEL_IO>
    ether 0a:e8:56:47:32:f8
    media: autoselect
    status: inactive
awdl0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1484
    options=400<CHANNEL_IO>
    ether 06:ff:ce:ce:60:5d
    inet6 fe80::4ff:ceff:fece:605d%awdl0 prefixlen 64 scopeid 0x9
    nd6 options=201<PERFORMNUD,DAD>
    media: autoselect
    status: active
llw0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    options=400<CHANNEL_IO>
    ether 06:ff:ce:ce:60:5d
    inet6 fe80::4ff:ceff:fece:605d%llw0 prefixlen 64 scopeid 0xa
    nd6 options=201<PERFORMNUD,DAD>
    media: autoselect
    status: active
utun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1380
    inet6 fe80::4571:abd6:537c:b14a%utun0 prefixlen 64 scopeid 0xb
    nd6 options=201<PERFORMNUD,DAD>
utun1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 2000
    inet6 fe80::7c38:d7:2cff:749c%utun1 prefixlen 64 scopeid 0xc
    nd6 options=201<PERFORMNUD,DAD>
en4: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
    options=400<CHANNEL_IO>
    ether 22:f3:ad:2c:85:a5
    media: autoselect
    status: active
bridge100: flags=8a63<UP,BROADCAST,SMART,RUNNING,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
    options=3<RXCSUM,TXCSUM>
    ether ba:e8:56:74:db:64
    inet 192.168.64.1 netmask 0xffffff00 broadcast 192.168.64.255
    inet6 fe80::1cd8:2f57:fc9c:3e3c%bridge100 prefixlen 64 secured scopeid 0xe
    Configuration:
        id 0:0:0:0:0:0 priority 0 hellotime 0 fwddelay 0
        maxage 0 holdcnt 0 proto stp maxaddr 100 timeout 1200
        root id 0:0:0:0:0:0 priority 0 ifcost 0 port 0
        ipfilter disabled flags 0x2
    member: en4 flags=3<LEARNING,DISCOVER>
            ifmaxaddr 0 port 13 priority 0 path cost 0
    nd6 options=201<PERFORMNUD,DAD>
    media: autoselect
    status: active

netstart -nr:

Internet:
Destination        Gateway            Flags        Netif Expire
default            172.30.4.1         UGSc           en0
default            link#14            UCSI       bridge1      !
127                127.0.0.1          UCS            lo0
127.0.0.1          127.0.0.1          UH             lo0
169.254            link#4             UCS            en0      !
172.30.4/24        link#4             UCS            en0      !
172.30.4.1/32      link#4             UCS            en0      !
172.30.4.1         0:27:d:59:e2:ca    UHLWIir        en0   1176
172.30.4.5         18:1d:ea:a5:cf:8f  UHLWI          en0   1143
172.30.4.7         4c:bb:58:f2:6f:43  UHLWI          en0   1081
172.30.4.13        9c:30:5b:b:d9:b7   UHLWI          en0   1200
172.30.4.62        0:28:f8:8a:b4:98   UHLWIi         en0    925
172.30.4.71        f8:da:c:30:7a:5b   UHLWI          en0    960
172.30.4.80/32     link#4             UCS            en0      !
172.30.4.87        d8:f:99:85:e7:2f   UHLWI          en0   1121
172.30.4.90        10:5b:ad:b2:cf:33  UHLWI          en0    882
172.30.4.122       b8:81:98:66:e2:57  UHLWI          en0    925
172.30.4.138       40:b8:9a:f6:19:7f  UHLWI          en0    952
172.30.4.140       9c:30:5b:d8:a7:85  UHLWI          en0    889
172.30.4.166       98:3b:8f:a8:cd:6b  UHLWI          en0   1024
172.30.4.209       54:62:e2:54:4d:5d  UHLWI          en0    879
172.30.4.210       a8:a7:95:7d:84:a7  UHLWI          en0    939
172.30.4.212       80:4a:14:94:8:ce   UHLWI          en0   1101
172.30.4.214       4c:bb:58:73:e7:fb  UHLWI          en0   1187
172.30.4.221       fc:1:7c:44:f7:bf   UHLWI          en0   1200
172.30.4.225       8c:86:1e:ca:35:d0  UHLWIi         en0    864
172.30.4.233       fc:1:7c:44:b8:e5   UHLWI          en0    960
172.30.4.252       5c:ea:1d:1d:6d:3b  UHLWI          en0   1057
192.168.64         link#14            UC         bridge1      !
192.168.64.2       2.4b.d8.ed.74.97   UHLWIi     bridge1   1096
224.0.0/4          link#4             UmCS           en0      !
224.0.0.251        1:0:5e:0:0:fb      UHmLWI         en0
239.255.255.250    1:0:5e:7f:ff:fa    UHmLWI         en0
255.255.255.255/32 link#4             UCS            en0      !

Internet6:
Destination                             Gateway                         Flags         Netif Expire
default                                 fe80::%utun0                    UGcI          utun0
default                                 fe80::%utun1                    UGcI          utun1
::1                                     ::1                             UHL             lo0
fe80::%lo0/64                           fe80::1%lo0                     UcI             lo0
fe80::1%lo0                             link#1                          UHLI            lo0
fe80::%en0/64                           link#4                          UCI             en0
fe80::815:6e24:96d7:61a1%en0            88:e9:fe:6f:b3:61               UHLWI           en0
fe80::142b:dda6:3e72:28f4%en0           b8:e8:56:47:32:f8               UHLI            lo0
fe80::%awdl0/64                         link#9                          UCI           awdl0
fe80::4ff:ceff:fece:605d%awdl0          6:ff:ce:ce:60:5d                UHLI            lo0
fe80::%llw0/64                          link#10                         UCI            llw0
fe80::4ff:ceff:fece:605d%llw0           6:ff:ce:ce:60:5d                UHLI            lo0
fe80::%utun0/64                         fe80::4571:abd6:537c:b14a%utun0 UcI           utun0
fe80::4571:abd6:537c:b14a%utun0         link#11                         UHLI            lo0
fe80::%utun1/64                         fe80::7c38:d7:2cff:749c%utun1   UcI           utun1
fe80::7c38:d7:2cff:749c%utun1           link#12                         UHLI            lo0
fe80::%bridge100/64                     link#14                         UCI        bridge10
fe80::1cd8:2f57:fc9c:3e3c%bridge100     ba.e8.56.74.db.64               UHLI            lo0
ff01::%lo0/32                           ::1                             UmCI            lo0
ff01::%en0/32                           link#4                          UmCI            en0
ff01::%awdl0/32                         link#9                          UmCI          awdl0
ff01::%llw0/32                          link#10                         UmCI           llw0
ff01::%utun0/32                         fe80::4571:abd6:537c:b14a%utun0 UmCI          utun0
ff01::%utun1/32                         fe80::7c38:d7:2cff:749c%utun1   UmCI          utun1
ff01::%bridge100/32                     link#14                         UmCI       bridge10
ff02::%lo0/32                           ::1                             UmCI            lo0
ff02::%en0/32                           link#4                          UmCI            en0
ff02::%awdl0/32                         link#9                          UmCI          awdl0
ff02::%llw0/32                          link#10                         UmCI           llw0
ff02::%utun0/32                         fe80::4571:abd6:537c:b14a%utun0 UmCI          utun0
ff02::%utun1/32                         fe80::7c38:d7:2cff:749c%utun1   UmCI          utun1
ff02::%bridge100/32                     link#14                         UmCI       bridge10

PS: I realise this is not really the place to ask for this kind of solutions, but I'm not sure where to ask for help either.

ricab commented 4 years ago

Hi @konart , wow a lot of stuff going on in there :slightly_smiling_face: I can't see multipass's bridge in there, I wonder why it went away.

Does a snap restart multipass.multipassd help? If not, what about snap disable multipass && snap enable multipass? Finally, if the issue persists, does uninstalling and reinstalling multipass help?

ricab commented 4 years ago

@konart since you said that things worked for a few seconds, could you please also note what ifconfig | grep mpqemubr says right after each of the steps above? Thanks and sorry you are having trouble with this.

konart commented 4 years ago

@ricab

Forgot to mention I'm on macOS, so I used

sudo launchctl load /Library/LaunchDaemons/com.canonical.multipassd.plist

sudo launchctl unload /Library/LaunchDaemons/com.canonical.multipassd.plist

to restart the daemon.

brew cask reinstall multipass does not help either

As for the ifconfig | grep mpqemubr - it returns nothing at any given step.

I've tried @Saviq https://discourse.ubuntu.com/t/troubleshooting-networking-on-macos/12901 guide (the 'Network routing problems' part as it seems to be my case), with no luck. (except for the 'configure Multipass to use a different subnet' - I'll give it a try a bit later this evening I guess)

Thank you for you time!

ricab commented 4 years ago

Ah I see. OK, I hope that helps!

konart commented 4 years ago

Alas.

I've tried changing the sunet to 192.168.66.0

ifconfig from the multipass instance:

enp0s2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.66.2  netmask 255.255.255.0  broadcast 192.168.66.255
        inet6 fe80::ccde:d8ff:fe96:bd7  prefixlen 64  scopeid 0x20<link>
        ether ce:de:d8:96:0b:d7  txqueuelen 1000  (Ethernet)
        RX packets 345  bytes 33208 (33.2 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 281  bytes 36944 (36.9 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 88  bytes 6728 (6.7 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 88  bytes 6728 (6.7 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

the host:

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
    options=1203<RXCSUM,TXCSUM,TXSTATUS,SW_TIMESTAMP>
    inet 127.0.0.1 netmask 0xff000000
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
    nd6 options=201<PERFORMNUD,DAD>
gif0: flags=8010<POINTOPOINT,MULTICAST> mtu 1280
stf0: flags=0<> mtu 1280
en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    options=400<CHANNEL_IO>
    ether b8:e8:56:47:32:f8
    inet6 fe80::c74:9e74:319c:2047%en0 prefixlen 64 secured scopeid 0x4
    inet 172.30.4.80 netmask 0xffffff00 broadcast 172.30.4.255
    nd6 options=201<PERFORMNUD,DAD>
    media: autoselect
    status: active
en1: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
    options=460<TSO4,TSO6,CHANNEL_IO>
    ether 82:0f:03:7c:6d:80
    media: autoselect <full-duplex>
    status: inactive
en2: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
    options=460<TSO4,TSO6,CHANNEL_IO>
    ether 82:0f:03:7c:6d:81
    media: autoselect <full-duplex>
    status: inactive
bridge0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    options=63<RXCSUM,TXCSUM,TSO4,TSO6>
    ether 82:0f:03:7c:6d:80
    Configuration:
        id 0:0:0:0:0:0 priority 0 hellotime 0 fwddelay 0
        maxage 0 holdcnt 0 proto stp maxaddr 100 timeout 1200
        root id 0:0:0:0:0:0 priority 0 ifcost 0 port 0
        ipfilter disabled flags 0x2
    member: en1 flags=3<LEARNING,DISCOVER>
            ifmaxaddr 0 port 5 priority 0 path cost 0
    member: en2 flags=3<LEARNING,DISCOVER>
            ifmaxaddr 0 port 6 priority 0 path cost 0
    nd6 options=201<PERFORMNUD,DAD>
    media: <unknown type>
    status: inactive
p2p0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 2304
    options=400<CHANNEL_IO>
    ether 0a:e8:56:47:32:f8
    media: autoselect
    status: inactive
awdl0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1484
    options=400<CHANNEL_IO>
    ether b6:13:73:e1:58:5d
    inet6 fe80::b413:73ff:fee1:585d%awdl0 prefixlen 64 scopeid 0x9
    nd6 options=201<PERFORMNUD,DAD>
    media: autoselect
    status: active
llw0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    options=400<CHANNEL_IO>
    ether b6:13:73:e1:58:5d
    inet6 fe80::b413:73ff:fee1:585d%llw0 prefixlen 64 scopeid 0xa
    nd6 options=201<PERFORMNUD,DAD>
    media: autoselect
    status: active
utun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1380
    inet6 fe80::19f9:c683:e3f4:1d99%utun0 prefixlen 64 scopeid 0xb
    nd6 options=201<PERFORMNUD,DAD>
utun1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 2000
    inet6 fe80::257e:5ad9:89a3:b3ac%utun1 prefixlen 64 scopeid 0xc
    nd6 options=201<PERFORMNUD,DAD>
en4: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
    options=400<CHANNEL_IO>
    ether 22:f3:ad:2c:85:a5
    media: autoselect
    status: active
bridge100: flags=8a63<UP,BROADCAST,SMART,RUNNING,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
    options=3<RXCSUM,TXCSUM>
    ether ba:e8:56:74:db:64
    inet 192.168.66.1 netmask 0xffffff00 broadcast 192.168.66.255
    inet6 fe80::1431:b55f:83a8:20dd%bridge100 prefixlen 64 secured scopeid 0xe
    Configuration:
        id 0:0:0:0:0:0 priority 0 hellotime 0 fwddelay 0
        maxage 0 holdcnt 0 proto stp maxaddr 100 timeout 1200
        root id 0:0:0:0:0:0 priority 0 ifcost 0 port 0
        ipfilter disabled flags 0x2
    member: en4 flags=3<LEARNING,DISCOVER>
            ifmaxaddr 0 port 13 priority 0 path cost 0
    nd6 options=201<PERFORMNUD,DAD>
    media: autoselect
    status: active

With no effect. Still can't ping anything but the host

multipass@k3s:~$ ping google.com
PING google.com (74.125.205.102) 56(84) bytes of data.
^C
--- google.com ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1031ms

multipass@k3s:~$ ping 74.125.205.102
PING 74.125.205.102 (74.125.205.102) 56(84) bytes of data.

--- 74.125.205.102 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3159ms

multipass@k3s:~$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.

--- 8.8.8.8 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3215ms
Saviq commented 4 years ago

Hi @konart all this suggests the VPN is messing with the routing somehow.

There is another issue related to this, which had a solution that worked at least for some people: https://github.com/CanonicalLtd/multipass/issues/495#issuecomment-448461250, maybe it helps you, too. Although there the problem only arose when turning the VPN on, I believe.

Have a go, you seem to have the utun* interfaces, too, maybe it does help.

konart commented 4 years ago

Yeah, I've seen this one. Doesn't work in my case. Tried both utun* interfaces with no luck. 10-15 seconds after relaunching multipass and starting the instance - ping goes nowhere.

As a matter of fact - I've tried clean OS install and this thing happends with any VPN client it seems (I've tried Tunnelblick too, for example). Once you install it - there is no other option but to reinstall the whole system. At least I haven't found any other way for now.

konart commented 4 years ago

After some investigation: on my colleague's machine (same macbook, same OS, same VPN etc) everything works and ifconfig gives the following:

n5: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
  ether 7e:7c:62:1b:d4:76
  media: autoselect
  status: active
bridge100: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
  options=3<RXCSUM,TXCSUM>
  ether 3a:c9:86:c4:32:64
  inet 192.168.64.1 netmask 0xffffff00 broadcast 192.168.64.255
  Configuration:
    id 0:0:0:0:0:0 priority 0 hellotime 0 fwddelay 0
    maxage 0 holdcnt 0 proto stp maxaddr 100 timeout 1200
    root id 0:0:0:0:0:0 priority 0 ifcost 0 port 0
    ipfilter disabled flags 0x2
  member: en5 flags=3<LEARNING,DISCOVER>
          ifmaxaddr 0 port 17 priority 0 path cost 0
  nd6 options=201<PERFORMNUD,DAD>
  media: autoselect
  status: active

where en5 as I understand is multipass' instance

here's my part of ifconfig:

en4: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
    options=400<CHANNEL_IO>
    ether e6:61:df:6a:d6:23
    media: autoselect
    status: active
bridge100: flags=8a63<UP,BROADCAST,SMART,RUNNING,ALLMULTI,SIMPLEX,MULTICAST> mtu 1500
    options=3<RXCSUM,TXCSUM>
    ether ba:e8:56:74:db:64
    inet 192.168.64.1 netmask 0xffffff00 broadcast 192.168.64.255
    inet6 fe80::14f7:15e3:3054:8c5a%bridge100 prefixlen 64 secured scopeid 0xe
    Configuration:
        id 0:0:0:0:0:0 priority 0 hellotime 0 fwddelay 0
        maxage 0 holdcnt 0 proto stp maxaddr 100 timeout 1200
        root id 0:0:0:0:0:0 priority 0 ifcost 0 port 0
        ipfilter disabled flags 0x2
    member: en4 flags=3<LEARNING,DISCOVER>
            ifmaxaddr 0 port 13 priority 0 path cost 0
    nd6 options=201<PERFORMNUD,DAD>
    media: autoselect
    status: active

Note this options=400<CHANNEL_IO> in en4 interface, that is a member of 'bridge100'.

So far this is the only thing I can find different between the two of us. Not sure how to configure it though.

zhuch-h commented 4 years ago

I have the same problem

zhuch-h commented 4 years ago

restart solve,ple refer to it. Duplicate of https://github.com/CanonicalLtd/multipass/issues/1025#issuecomment-528572275#

konart commented 4 years ago

I may assure you that I've tried restarting both multipass and the whole machine more than once since the installation. In fact I've even tried clean macOS restart.

The problem is the VPN here. Somehow it does not allow multipass to work as intended. (see previous comments)

jwthomp commented 4 years ago

I wanted to add a little bit of information to this that might be valuable. I am having the same issue whenever I run pritunl (openvpn) on my mac. multipass networking works fine until I turn it on (and works again of I turn it off).

One thing I noticed is that multipass VPN's seem to be sending their traffic out twice whenever using ICMP or TCP. As an example here are the results of doing a ping inside a multipass VPN when pritunl is disabled. The ping works.

multipass@k3s:~$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=52 time=6.56 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=52 time=7.51 ms

A tcpdump on my machine shows:

$ sudo tcpdump -n icmp
tcpdump: data link type PKTAP
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on pktap, link-type PKTAP (Apple DLT_PKTAP), capture size 262144 bytes
13:55:17.649221 IP 192.168.64.3 > 8.8.8.8: ICMP echo request, id 3100, seq 1, length 64
13:55:17.649245 IP 10.150.30.149 > 8.8.8.8: ICMP echo request, id 2646, seq 1, length 64
13:55:17.655575 IP 8.8.8.8 > 10.150.30.149: ICMP echo reply, id 2646, seq 1, length 64
13:55:17.655605 IP 8.8.8.8 > 192.168.64.3: ICMP echo reply, id 3100, seq 1, length 64
13:55:18.718736 IP 192.168.64.3 > 8.8.8.8: ICMP echo request, id 3100, seq 2, length 64
13:55:18.718806 IP 10.150.30.149 > 8.8.8.8: ICMP echo request, id 2646, seq 2, length 64
13:55:18.725815 IP 8.8.8.8 > 10.150.30.149: ICMP echo reply, id 2646, seq 2, length 64
13:55:18.725851 IP 8.8.8.8 > 192.168.64.3: ICMP echo reply, id 3100, seq 2, length 64

I see similar results if I try to telnet to www.google.com:80. I'll see a tcp syn going out over the wire from both source IP's (192.168.64.3 and 10.150.30.149).

If I run the above from the docker for mac vm I only see packets coming from 10.150.30.149

$ docker run -it --rm --privileged --pid=host debian nsenter -t 1 -m -u -n -i sh
/ # ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=38 time=9.721 ms
64 bytes from 8.8.8.8: seq=1 ttl=38 time=8.740 ms

and again from tcpdump I only see the one IP this time:

14:00:00.768940 IP 10.150.30.149 > 8.8.8.8: ICMP echo request, id 0, seq 0, length 64
14:00:00.777263 IP 8.8.8.8 > 10.150.30.149: ICMP echo reply, id 0, seq 0, length 64
14:00:01.768313 IP 10.150.30.149 > 8.8.8.8: ICMP echo request, id 0, seq 1, length 64
14:00:01.776125 IP 8.8.8.8 > 10.150.30.149: ICMP echo reply, id 0, seq 1, length 64

Here are my routing tables with VPN turned OFF:

$ netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags        Refs      Use   Netif Expire
default            10.150.30.1        UGSc          100     5530     en0
default            link#25            UCSI            1        0 bridge1      !
10.150.30/24       link#8             UCS             7        0     en0      !
10.150.30.1/32     link#8             UCS             1        0     en0      !
10.150.30.1        0:86:9c:74:bd:11   UHLWIir        35       80     en0   1043
10.150.30.3        8c:85:90:d2:ca:54  UHLWI           0        0     en0   1112
10.150.30.149/32   link#8             UCS             0        0     en0      !
10.150.30.151      8c:85:90:9c:3e:b5  UHLWI           0        0     en0    305
10.150.30.159      8c:85:90:2a:f3:4b  UHLWI           0       21     en0    216
10.150.30.167      f0:18:98:48:3b:9d  UHLWI           0        1     en0    209
10.150.30.235      d4:61:9d:0:62:e2   UHLWI           0        3     en0    690
10.150.30.241      18:1d:ea:db:ec:f5  UHLWI           0        0     en0    236
127                127.0.0.1          UCS             0        0     lo0
127.0.0.1          127.0.0.1          UH            263   266820     lo0
127.94.0.1         127.94.0.1         UH              0        0     lo0
127.94.0.2         127.94.0.2         UH              0        0     lo0
127.94.0.3         127.94.0.3         UH              0        0     lo0
169.254            link#8             UCS             2        0     en0      !
169.254            link#22            UCSI            2        0     en7      !
169.254.6.16       f0:18:98:6:7e:47   UHLSW           0        1     en0      !
169.254.126.244    c6:61:8b:44:22:8b  UHLSW           3      150     en7    499
169.254.139.62     c6:61:8b:44:22:8d  UHLSW           0        5     lo0
169.254.139.62/32  link#22            UCS             0        0     en7      !
192.168.64         link#25            UC              2        0 bridge1      !
192.168.64.1       8e.85.90.a2.56.64  UHLWI           0        6     lo0
192.168.64.3       5e.f9.ef.d9.37.4c  UHLWIi          3    14386 bridge1   1098
224.0.0/4          link#8             UmCS            2        0     en0      !
224.0.0/4          link#22            UmCSI           2        0     en7      !
224.0.0.251        1:0:5e:0:0:fb      UHmLWI          0        0     en0
224.0.0.251        1:0:5e:0:0:fb      UHmLWI          0        0     en7
224.0.0.251        1:0:5e:0:0:fb      UHmLWI          0        0 bridge1
239.255.255.250    1:0:5e:7f:ff:fa    UHmLWI          0     1161     en0
239.255.255.250    1:0:5e:7f:ff:fa    UHmLWI          0       93     en7
255.255.255.255/32 link#8             UCS             0        0     en0      !
255.255.255.255/32 link#22            UCSI            0        0     en7      !

Internet6:
Destination                             Gateway                         Flags         Netif Expire
default                                 fe80::%utun0                    UGcI          utun0
default                                 fe80::%utun1                    UGcI          utun1
default                                 fe80::%utun2                    UGcI          utun2
::1                                     ::1                             UHL             lo0
fe80::%lo0/64                           fe80::1%lo0                     UcI             lo0
fe80::1%lo0                             link#1                          UHLI            lo0
fe80::%en5/64                           link#7                          UCI             en5
fe80::aede:48ff:fe00:1122%en5           ac:de:48:0:11:22                UHLI            lo0
fe80::aede:48ff:fe33:4455%en5           ac:de:48:33:44:55               UHLWIi          en5
fe80::%en0/64                           link#8                          UCI             en0
fe80::420:cf91:2d75:328d%en0            8c:85:90:2a:fb:e9               UHLI            lo0
fe80::42a:708e:ccf7:f09a%en0            8c:85:90:2a:f3:4b               UHLWI           en0
fe80::465:4815:9492:c36%en0             d4:61:9d:b:d2:44                UHLWI           en0
fe80::cd7:e595:65d8:1ccf%en0            f0:18:98:1b:cf:d4               UHLWI           en0
fe80::cdb:9fdf:ca29:14ea%en0            8c:85:90:bf:10:1c               UHLWI           en0
fe80::10e5:2d2f:1997:36f3%en0           48:bf:6b:f1:5:5e                UHLWI           en0
fe80::1402:ca1d:6821:151e%en0           7c:4:d0:ce:c5:10                UHLWI           en0
fe80::1413:c870:abb1:26a0%en0           f0:18:98:b3:21:6                UHLWI           en0
fe80::141a:5f78:9654:c7c%en0            38:f9:d3:48:9c:77               UHLWI           en0
fe80::14ab:e04b:97d:430c%en0            8c:85:90:99:10:28               UHLWI           en0
fe80::185c:40db:b7d5:22d6%en0           38:f9:d3:ad:f0:29               UHLWI           en0
fe80::%awdl0/64                         link#14                         UCI           awdl0
fe80::58ef:32ff:fe0c:5c39%awdl0         5a:ef:32:c:5c:39                UHLI            lo0
fe80::%utun0/64                         fe80::986a:d852:9139:e940%utun0 UcI           utun0
fe80::986a:d852:9139:e940%utun0         link#16                         UHLI            lo0
fe80::%utun1/64                         fe80::35a3:ff47:175d:231b%utun1 UcI           utun1
fe80::35a3:ff47:175d:231b%utun1         link#17                         UHLI            lo0
fe80::%utun2/64                         fe80::d8a7:18eb:c934:a753%utun2 UcI           utun2
fe80::d8a7:18eb:c934:a753%utun2         link#18                         UHLI            lo0
fe80::%en7/64                           link#22                         UCI             en7
fe80::10b0:e1e5:1f57:ad2%en7            c6:61:8b:44:22:8d               UHLI            lo0
ff01::%lo0/32                           ::1                             UmCI            lo0
ff01::%en5/32                           link#7                          UmCI            en5
ff01::%en0/32                           link#8                          UmCI            en0
ff01::%awdl0/32                         link#14                         UmCI          awdl0
ff01::%utun0/32                         fe80::986a:d852:9139:e940%utun0 UmCI          utun0
ff01::%utun1/32                         fe80::35a3:ff47:175d:231b%utun1 UmCI          utun1
ff01::%utun2/32                         fe80::d8a7:18eb:c934:a753%utun2 UmCI          utun2
ff01::%en7/32                           link#22                         UmCI            en7
ff02::%lo0/32                           ::1                             UmCI            lo0
ff02::%en5/32                           link#7                          UmCI            en5
ff02::%en0/32                           link#8                          UmCI            en0
ff02::%awdl0/32                         link#14                         UmCI          awdl0
ff02::%utun0/32                         fe80::986a:d852:9139:e940%utun0 UmCI          utun0
ff02::%utun1/32                         fe80::35a3:ff47:175d:231b%utun1 UmCI          utun1
ff02::%utun2/32                         fe80::d8a7:18eb:c934:a753%utun2 UmCI          utun2
ff02::%en7/32                           link#22                         UmCI            en7

If I enable the VPN then networking from the multipass vm stops working. The docker vm can still route out. Please let me know if I can provide any other information that might be helpful.

Note: I don't believe bridge100 is being lost (per above comment), I think netstat -rn is just clipping it down to bridge1.

MindTooth commented 4 years ago

This might be related. As I am myself using DNSCrypt-Proxy, I seem unable to use multipass (ping urls from inside a vm).

ashee commented 4 years ago

In my case, I was running homebrew dnsmasq which was interfering with DNS. https://discourse.ubuntu.com/t/troubleshooting-networking-on-macos/12901

$ sudo lsof -iTCP:53 -iUDP:53 -n -P

In my case, instead of mDNSResponder, it was reporting dnsmasq

After stopping dnsmasq, guest network in microk8s-vm started working

$ sudo launchctl unload /Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist
nmpacheco commented 4 years ago

This issue has not been updated for a while but let me add that I have the exact same issue and I have noticed the most weird pattern. A while after rebooting the mac (some days, don't know exactly how many) the connectivity from multipass vm to the external world starts working again. I have openvpn and forticlient installed. Except for reinstalling mac, I have tried all the things that are suggested in the several issues and troubleshooting links with no luck. The only thing that works is waiting patiently for some days and eventually the connectivity magically reappears... Needless to say, I gave up on multipass as a reliable solution for the tests I need to be constantly running...

noomz commented 4 years ago

I use TunnelBear and this comment works!

I added nat on utun1 from bridge100:network to any -> (utun1) to file /etc/pf.conf. Then I reloaded the file: $ sudo pfctl -f /etc/pf.conf.

https://github.com/canonical/multipass/issues/495#issuecomment-448461250

frarito commented 3 years ago

Any update @konart ? I'm in a similar situation.. tried all alternatives and still can't make it work

noomz commented 3 years ago

@frarito did you try changing utun1 to something matched your ifconfig ? I found that it changed from time to time.

angelajfisher commented 3 years ago

I just want to jump in here for the possible benefit of any future people who were just as frustrated & confused by this issue as I was. What I found that worked for me was deleting the VPN's configs through the terminal!

In my case, I was trying to uninstall NordVPN, so I found its files by entering defaults read and then command+F'ing for "nord". Once I found the corresponding files, I simply entered defaults delete <filename> for every relevant instance and thankfully after a computer restart all was fixed!!

I really hope this helps someone keep from having to reinstall their entire OS, as I've seen many suggest. Best of luck in your troubleshooting!

nikitalpopov commented 3 years ago

As a matter of fact - I've tried clean OS install and this thing happends with any VPN client it seems (I've tried Tunnelblick too, for example). Once you install it - there is no other option but to reinstall the whole system. At least I haven't found any other way for now.

Actually, you don't have to reinstall the whole system. I removed Checkpoint with this command and everything started to work:

> sudo /Library/Application\ Support/Checkpoint/Endpoint\ Connect/uninstall