search

canonical / multipass

Multipass orchestrates virtual Ubuntu instances
https://multipass.run
GNU General Public License v3.0
7.87k stars 650 forks source link

Multipass unable to connect and retrieve images #3662

Open mcw-work opened 2 months ago

mcw-work commented 2 months ago

Describe the bug When starting up the Multipass snap, it gives a message of "Failed to retrieve images: failed to download from 'https://cdimage.ubuntu.com/..... ": Operation Cancelled.

As similar message appears when using the multipass cli.

To Reproduce Just run multipass

Expected behavior A list of images of instances I can create.

Logs Please provide logs from the daemon, see accessing logs on where to find them on your platform.

Additional info

Additional context It seemed to be working fine and then just stopped. URLs can be accessed from a browser with no issue. There is no VPN or firewall in place and snap interfaces all appear connected. Log file attached.

multipass.log

ricab commented 2 months ago

Hi @mcw-work, from your logs, Multipass just couldn't resolve or get to anything on the outside:

Host cloud-images.ubuntu.com not found
Host codeload.github.com not found 
Host cdimage.ubuntu.com not found

I would ask for ufw status or any funny DNS setup, but you've already told me it is all pretty vanilla. Those errors are later replaced with network timeouts... Not sure what is going on, but something is preventing your Multipass from getting to the interwebs.

I wonder if there was some transient issue with your ISP that was limited to IPv4 (your browsed could have succeeded on IPv6)? If the issue remains, could you please try

  1. curl with IPv4/6 to see if it has the same issue?
  2. snap run --shell multipass.multipassd
    1. getent ahosts cloud-images.ubuntu.com
    2. ping the IPs that you get that way
    3. curl again on the inside
mcw-work commented 2 months ago

No problem: ufw status shows as inactive

So, it does seem my ISP is only providing IPv4. Curling works with -4 but not -6. Running the snap shell, I can ping the ipv4 addresses but not the ipv6 ones. Also, I can't curl from inside the snap as it seems curl wasn't packaged.

Annoying my mobile phone network also seems to not permit IPv6 so I will have to find another network that does support it to see if that is the issue. It seems strange that it needs IPv6 to work though.

ricab commented 2 months ago

Also, I can't curl from inside the snap as it seems curl wasn't packaged.

Right, I overlooked that. What does ping -4/-6 say though?

So, it does seem my ISP is only providing IPv4. Curling works with -4 but not -6.

OK, so disabling IPv6 in your browser does not make a difference there, right? It can still download fine?

It seems strange that it needs IPv6 to work though.

Indeed. Nothing we've noticed before... FWIW, I just disabled IPv6 on my host entirely and it works just fine, i.e. no trouble pinging IPv4 from within the snap shell and no complaints from Multipass.

mcw-work commented 2 months ago

Disabling IPv6 in browser makes no difference - I can still download. 

mikec-w@mcwlaptop:~$ ping -4 -c 2 cloud-images.ubuntu.com
PING cloud-images.ubuntu.com (185.125.190.40) 56(84) bytes of data.
64 bytes from https-services.aerodent.canonical.com (185.125.190.40): icmp_seq=1 ttl=55 time=13.4 ms
64 bytes from https-services.aerodent.canonical.com (185.125.190.40): icmp_seq=2 ttl=55 time=13.6 ms

--- cloud-images.ubuntu.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 13.379/13.488/13.598/0.109 ms
mikec-w@mcwlaptop:~$ ping -6 -c 2 cloud-images.ubuntu.com
PING cloud-images.ubuntu.com (2620:2d:4000:1::1a) 56 data bytes
From 2a01:4b00:bc15:e100:4aed:e6ff:fe8a:a130 icmp_seq=1 Destination unreachable: No route
From 2a01:4b00:bc15:e100:4aed:e6ff:fe8a:a130 icmp_seq=2 Destination unreachable: No route

--- cloud-images.ubuntu.com ping statistics ---
2 packets transmitted, 0 received, +2 errors, 100% packet loss, time 1001ms

mikec-w@mcwlaptop:~$

I think the lack of IPv4 might be a red-herring

ricab commented 2 months ago

Yeah probably. Is that output from inside the snap shell? Have you tried the getent command above? And what does ip route say? How does it compare inside and outside of the snap shell?

mcw-work commented 2 months ago

Hi, yes those pings are from within the snap shell.

Getent output from inside snap shell is as follows:

mikec-w@mcwlaptop:/home/mikec-w$ getent ahosts cloud-images.ubuntu.com
2620:2d:4000:1::1a STREAM cloud-images.ubuntu.com
2620:2d:4000:1::1a DGRAM  
2620:2d:4000:1::1a RAW    
2620:2d:4000:1::17 STREAM 
2620:2d:4000:1::17 DGRAM  
2620:2d:4000:1::17 RAW    
185.125.190.37  STREAM 
185.125.190.37  DGRAM  
185.125.190.37  RAW    
185.125.190.40  STREAM 
185.125.190.40  DGRAM  
185.125.190.40  RAW    
mikec-w@mcwlaptop:/home/mikec-w$

and the ip route:

default via 192.168.1.1 dev wlp0s20f3 proto dhcp src 192.168.1.154 metric 600 
10.12.44.0/24 dev lxdbr0 proto kernel scope link src 10.12.44.1 linkdown 
10.20.178.0/24 dev mpqemubr0 proto kernel scope link src 10.20.178.1 linkdown 
10.41.246.0/24 dev mpbr0 proto kernel scope link src 10.41.246.1 linkdown 
192.168.1.0/24 dev wlp0s20f3 proto kernel scope link src 192.168.1.154 metric 600 
mikec-w@mcwlaptop:/home/mikec-w$

Does that help at all?

giuliazanchi commented 1 month ago

Adding the "documentation" label to keep an eye on suggested troubleshooting or workarounds.

ricab commented 1 month ago

Hi @mcw-work, sorry for the late reply.

Does that help at all?

Your output is an indication that the problem lies in the Multipass daemon itself and not snap confinement. But we don't have much more than that at this point. Our guess would be some bug in the Qt code we use to download, but hard to say without a reproducer to debug.

ricab commented 1 month ago

@s-makin, to try to exclude a relation with that, do you have IPv6 in your network? What do the following commands say for you?

ping -4 -c 2 cloud-images.ubuntu.com
ping -6 -c 2 cloud-images.ubuntu.com

Thank you.

s-makin commented 1 month ago

ping -4 -c 2 cloud-images.ubuntu.com

PING  (185.125.190.37) 56(84) bytes of data.
64 bytes from https-services.actiontoad.canonical.com (185.125.190.37): icmp_seq=1 ttl=58 time=6.01 ms
64 bytes from https-services.actiontoad.canonical.com (185.125.190.37): icmp_seq=2 ttl=58 time=6.67 ms

---  ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 6.007/6.339/6.672/0.332 ms

ping -6 -c 2 cloud-images.ubuntu.com

PING cloud-images.ubuntu.com(https-services.actiontoad.canonical.com (2620:2d:4000:1::17)) 56 data bytes

--- cloud-images.ubuntu.com ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1017ms
ricab commented 1 month ago

Ahha, so no IPv6 for you either, it looks like. I wonder if Qt tries IPv6 and aborts if that doesn't work...

s-makin commented 1 month ago

Ahha, so no IPv6 for you either, it looks like. I wonder if Qt tries IPv6 and aborts if that doesn't work...

It's strange that it does sometimes work then - I managed to successfully launch one instance today, and last week (and before that) I didn't have this problem at all.

ricab commented 1 month ago

If the (vague) hypothesis above held, and it sometimes tried IPv4 first for some reason, then it would work on those occasions.

BTW, does disabling IPv6 in Ubuntu help? (You probaby want the sections for Netplan/NetworkManager in that document.)

mcw-work commented 1 month ago

Hi @ricab - no disabling IPv6 in Ubuntu doesn't help but I think you may be onto the right path. Checking my router this evening it does not have an IPv6 address on its WAN connection. Thinking back this may have stopped working after coming home from another house that had a different ISP. That one I am pretty sure has IPv6.

Annoyingly I can't test it on my mobile phone as that network doesn't give an IPv6 address either..... but tomorrow evening I will return to the other house. If the theory is correct - that will magically make it work. Doesn't solve the problem but it may confirm the hypothesis.

In the meantime I will also be pestering my ISP for an IPv6 address. I will report back tomorrow.

s-makin commented 1 month ago

If the (vague) hypothesis above held, and it sometimes tried IPv4 first for some reason, then it would work on those occasions.

BTW, does disabling IPv6 in Ubuntu help? (You probaby want the sections for Netplan/NetworkManager in that document.)

Disabling IPv6 didn't work for me either. Still get the exact same error.

andrei-toterman commented 1 month ago

@s-makin just out of curiosity, when you do something like multipass launch or multipass find --force-update and you get an error, do you get the error instantly or does the command block for a while before returning that error?

mcw-work commented 1 month ago

Instantly for me.

s-makin commented 1 month ago

@s-makin just out of curiosity, when you do something like multipass launch or multipass find --force-update and you get an error, do you get the error instantly or does the command block for a while before returning that error?

Instantly, usually. There's been a couple of times where it's thought about it first and then returned the error, but mostly just instantaneous.

mcw-work commented 1 month ago

Ok, so I was right - I'm now on an internet connection that has proper IPv6 connectivity and, initially, it did the same thing. Then following a "snap restart multipass.multipassd" it has magically come to life and is all working.

Seems to definitely be an IPv6 related issue.

levkropp commented 1 month ago

Hi @mcw-work and @s-makin ,

We've been looking into this issue more but haven't had success in reproducing it, even when testing with routers that have IPv6 enabled and disabled. However, we've observed some additional strange behavior, such as the error occurring instantly when there should be a timeout.

At this point, we believe the issue may be related to the Qt network library we are using, so we're exploring alternative implementations for handling network connections. We're also doing additional testing using curl in the snap.

Since replicating the issue has been challenging, we're working on an alternative method for connecting and retrieving images, which we hope to share soon for further debugging.

Thanks for your patience as we continue investigating this!

mcw-work commented 1 month ago

No problem. Having returned to the original network connection - it is broken again. I am not sure how helpful it is, but my laptop has IPv6 from the router (and DHCP server) but the router itself has no IPv6 on the WAN side.

Let me know if you want me to try anything else to help recreate.

ricab commented 1 month ago

A couple of wild shots, but:

ricab commented 1 month ago

One more: does the following help?

snap stop multipass
sudo rm -rf /var/snap/multipass/common/cache/multipassd/network-cache/
snap start multipass
levkropp commented 1 month ago

We've packaged curl into a Multipass snap if you'd like to try running curl from inside the snap and letting us know if there are any unexpected results https://send.bitwarden.com/#OmPpuMuBDESpErH4AVE9Og/l4JIgd5NMjVO0cWt6xk76g

You can install it with the command sudo snap install multipass.snap --devmode --dangerous

Do you remember if getent ahosts cloud-images.ubuntu.com returned any IPv6 values when you ran it on the network without IPv6? In my testing I only saw IPv4 values returned when on an IPv4 only network.

s-makin commented 1 month ago 
* do you happen to have other VM/Containerization software installed?

I've played around with most of them at one point or another, but not recently - libvirt, qemu, lxd, etc. I don't use Docker (and have never installed it). 

  * `sudo iptables-save`
# Generated by iptables-save v1.8.7 on Sat Sep 28 11:38:04 2024
*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:LIBVIRT_PRT - [0:0]
-A POSTROUTING -j LIBVIRT_PRT
-A LIBVIRT_PRT -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
COMMIT
# Completed on Sat Sep 28 11:38:04 2024
# Generated by iptables-save v1.8.7 on Sat Sep 28 11:38:04 2024
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:LIBVIRT_FWI - [0:0]
:LIBVIRT_FWO - [0:0]
:LIBVIRT_FWX - [0:0]
:LIBVIRT_INP - [0:0]
:LIBVIRT_OUT - [0:0]
-A INPUT -j LIBVIRT_INP
-A FORWARD -j LIBVIRT_FWX
-A FORWARD -j LIBVIRT_FWI
-A FORWARD -j LIBVIRT_FWO
-A OUTPUT -j LIBVIRT_OUT
-A LIBVIRT_FWI -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A LIBVIRT_FWI -o virbr0 -j REJECT --reject-with icmp-port-unreachable
-A LIBVIRT_FWO -s 192.168.122.0/24 -i virbr0 -j ACCEPT
-A LIBVIRT_FWO -i virbr0 -j REJECT --reject-with icmp-port-unreachable
-A LIBVIRT_FWX -i virbr0 -o virbr0 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 67 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p tcp -m tcp --dport 68 -j ACCEPT
COMMIT
# Completed on Sat Sep 28 11:38:04 2024
# Generated by iptables-save v1.8.7 on Sat Sep 28 11:38:04 2024
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:LIBVIRT_PRT - [0:0]
-A POSTROUTING -j LIBVIRT_PRT
-A LIBVIRT_PRT -s 192.168.122.0/24 -d 224.0.0.0/24 -j RETURN
-A LIBVIRT_PRT -s 192.168.122.0/24 -d 255.255.255.255/32 -j RETURN
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE
COMMIT
# Completed on Sat Sep 28 11:38:04 2024
  * `sudo iptables-legacy-save`

Doesn't output anything

  * `sudo iptables-nft-save`
# Generated by iptables-nft-save v1.8.7 on Sat Sep 28 11:41:08 2024
*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:LIBVIRT_PRT - [0:0]
-A POSTROUTING -j LIBVIRT_PRT
-A LIBVIRT_PRT -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
COMMIT
# Completed on Sat Sep 28 11:41:08 2024
# Generated by iptables-nft-save v1.8.7 on Sat Sep 28 11:41:08 2024
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:LIBVIRT_FWI - [0:0]
:LIBVIRT_FWO - [0:0]
:LIBVIRT_FWX - [0:0]
:LIBVIRT_INP - [0:0]
:LIBVIRT_OUT - [0:0]
-A INPUT -j LIBVIRT_INP
-A FORWARD -j LIBVIRT_FWX
-A FORWARD -j LIBVIRT_FWI
-A FORWARD -j LIBVIRT_FWO
-A OUTPUT -j LIBVIRT_OUT
-A LIBVIRT_FWI -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A LIBVIRT_FWI -o virbr0 -j REJECT --reject-with icmp-port-unreachable
-A LIBVIRT_FWO -s 192.168.122.0/24 -i virbr0 -j ACCEPT
-A LIBVIRT_FWO -i virbr0 -j REJECT --reject-with icmp-port-unreachable
-A LIBVIRT_FWX -i virbr0 -o virbr0 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p udp -m udp --dport 67 -j ACCEPT
-A LIBVIRT_INP -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT
-A LIBVIRT_OUT -o virbr0 -p tcp -m tcp --dport 68 -j ACCEPT
COMMIT
# Completed on Sat Sep 28 11:41:08 2024
# Generated by iptables-nft-save v1.8.7 on Sat Sep 28 11:41:08 2024
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:LIBVIRT_PRT - [0:0]
-A POSTROUTING -j LIBVIRT_PRT
-A LIBVIRT_PRT -s 192.168.122.0/24 -d 224.0.0.0/24 -j RETURN
-A LIBVIRT_PRT -s 192.168.122.0/24 -d 255.255.255.255/32 -j RETURN
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
-A LIBVIRT_PRT -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE
COMMIT
# Completed on Sat Sep 28 11:41:08 2024
s-makin commented 1 month ago

One more: does the following help?

snap stop multipass
sudo rm -rf /var/snap/multipass/common/cache/multipassd/network-cache/
snap start multipass

This didn't do anything - still get the same launch failed: Remote "" is unknown or unreachable as before

s-makin commented 1 month ago

We've packaged curl into a Multipass snap if you'd like to try running curl from inside the snap and letting us know if there are any unexpected results https://send.bitwarden.com/#OmPpuMuBDESpErH4AVE9Og/l4JIgd5NMjVO0cWt6xk76g

You can install it with the command sudo snap install multipass.snap --devmode --dangerous

Do you remember if getent ahosts cloud-images.ubuntu.com returned any IPv6 values when you ran it on the network without IPv6? In my testing I only saw IPv4 values returned when on an IPv4 only network.

Thanks, I'll try the first part on Monday. This is what I get as output when I run getent ahosts cloud-images.ubuntu.com before disabling IPv6:

2620:2d:4000:1::1a STREAM cloud-images.ubuntu.com
2620:2d:4000:1::1a DGRAM  
2620:2d:4000:1::1a RAW    
2620:2d:4000:1::17 STREAM 
2620:2d:4000:1::17 DGRAM  
2620:2d:4000:1::17 RAW    
185.125.190.37  STREAM 
185.125.190.37  DGRAM  
185.125.190.37  RAW    
185.125.190.40  STREAM 
185.125.190.40  DGRAM  
185.125.190.40  RAW

And this is what I get after disabling it (if it helps):

2620:2d:4000:1::17 STREAM cloud-images.ubuntu.com
2620:2d:4000:1::17 DGRAM  
2620:2d:4000:1::17 RAW    
2620:2d:4000:1::1a STREAM 
2620:2d:4000:1::1a DGRAM  
2620:2d:4000:1::1a RAW    
185.125.190.37  STREAM 
185.125.190.37  DGRAM  
185.125.190.37  RAW    
185.125.190.40  STREAM 
185.125.190.40  DGRAM  
185.125.190.40  RAW
mcw-work commented 1 month ago

Same for me - clearing the cache has not effect (although it did take a bit longer for the error to come up this time).

Other VMs - I have LXD installed but that's it. It would also seem that bitwarden file send doesn't work - it just spins for me when I click the download button for the snap.

As for the IPtables output. 

mikec-w@mcwlaptop:~$ sudo iptables-save
# Generated by iptables-save v1.8.10 (nf_tables) on Sat Sep 28 13:00:25 2024
*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING -o mpqemubr0 -p udp -m udp --dport 68 -m comment --comment "generated for Multipass network mpqemubr0" -j CHECKSUM --checksum-fill
COMMIT
# Completed on Sat Sep 28 13:00:25 2024
# Generated by iptables-save v1.8.10 (nf_tables) on Sat Sep 28 13:00:25 2024
*filter
:INPUT ACCEPT [18476942:45351479219]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [12661256:8498373490]
-A INPUT -i mpqemubr0 -p tcp -m tcp --dport 53 -m comment --comment "generated for Multipass network mpqemubr0" -j ACCEPT
-A INPUT -i mpqemubr0 -p udp -m udp --dport 53 -m comment --comment "generated for Multipass network mpqemubr0" -j ACCEPT
-A INPUT -i mpqemubr0 -p udp -m udp --dport 67 -m comment --comment "generated for Multipass network mpqemubr0" -j ACCEPT
-A FORWARD -i mpqemubr0 -o mpqemubr0 -m comment --comment "generated for Multipass network mpqemubr0" -j ACCEPT
-A FORWARD -s 10.20.178.0/24 -i mpqemubr0 -m comment --comment "generated for Multipass network mpqemubr0" -j ACCEPT
-A FORWARD -d 10.20.178.0/24 -o mpqemubr0 -m conntrack --ctstate RELATED,ESTABLISHED -m comment --comment "generated for Multipass network mpqemubr0" -j ACCEPT
-A FORWARD -i mpqemubr0 -m comment --comment "generated for Multipass network mpqemubr0" -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -o mpqemubr0 -m comment --comment "generated for Multipass network mpqemubr0" -j REJECT --reject-with icmp-port-unreachable
-A OUTPUT -o mpqemubr0 -p tcp -m tcp --sport 53 -m comment --comment "generated for Multipass network mpqemubr0" -j ACCEPT
-A OUTPUT -o mpqemubr0 -p udp -m udp --sport 53 -m comment --comment "generated for Multipass network mpqemubr0" -j ACCEPT
-A OUTPUT -o mpqemubr0 -p udp -m udp --sport 67 -m comment --comment "generated for Multipass network mpqemubr0" -j ACCEPT
COMMIT
# Completed on Sat Sep 28 13:00:25 2024
# Generated by iptables-save v1.8.10 (nf_tables) on Sat Sep 28 13:00:25 2024
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING -s 10.20.178.0/24 ! -d 10.20.178.0/24 -m comment --comment "generated for Multipass network mpqemubr0" -j MASQUERADE
-A POSTROUTING -s 10.20.178.0/24 ! -d 10.20.178.0/24 -p udp -m comment --comment "generated for Multipass network mpqemubr0" -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 10.20.178.0/24 ! -d 10.20.178.0/24 -p tcp -m comment --comment "generated for Multipass network mpqemubr0" -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 10.20.178.0/24 -d 255.255.255.255/32 -m comment --comment "generated for Multipass network mpqemubr0" -j RETURN
-A POSTROUTING -s 10.20.178.0/24 -d 224.0.0.0/24 -m comment --comment "generated for Multipass network mpqemubr0" -j RETURN
COMMIT
# Completed on Sat Sep 28 13:00:25 2024
# Warning: iptables-legacy tables present, use iptables-legacy-save to see them

mikec-w@mcwlaptop:~$ sudo iptables-legacy-save
# Generated by iptables-save v1.8.10 on Sat Sep 28 13:00:40 2024
*raw
:PREROUTING ACCEPT [19860399:49622223971]
:OUTPUT ACCEPT [12661354:8498393485]
COMMIT
# Completed on Sat Sep 28 13:00:40 2024
# Generated by iptables-save v1.8.10 on Sat Sep 28 13:00:40 2024
*mangle
:PREROUTING ACCEPT [19860399:49622223971]
:INPUT ACCEPT [18477018:45351489176]
:FORWARD ACCEPT [1382671:4270767284]
:OUTPUT ACCEPT [12661354:8498393485]
:POSTROUTING ACCEPT [14046601:12769431245]
COMMIT
# Completed on Sat Sep 28 13:00:40 2024
# Generated by iptables-save v1.8.10 on Sat Sep 28 13:00:40 2024
*nat
:PREROUTING ACCEPT [12380:3405711]
:INPUT ACCEPT [11204:3301655]
:OUTPUT ACCEPT [125629:9478357]
:POSTROUTING ACCEPT [124276:9269781]
COMMIT
# Completed on Sat Sep 28 13:00:40 2024
# Generated by iptables-save v1.8.10 on Sat Sep 28 13:00:40 2024
*filter
:INPUT ACCEPT [18477018:45351489176]
:FORWARD ACCEPT [1382671:4270767284]
:OUTPUT ACCEPT [12661354:8498393485]
COMMIT
# Completed on Sat Sep 28 13:00:40 2024

mikec-w@mcwlaptop:~$ sudo iptables-nft-save
# Generated by iptables-nft-save v1.8.10 (nf_tables) on Sat Sep 28 13:01:01 2024
*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING -o mpqemubr0 -p udp -m udp --dport 68 -m comment --comment "generated for Multipass network mpqemubr0" -j CHECKSUM --checksum-fill
COMMIT
# Completed on Sat Sep 28 13:01:01 2024
# Generated by iptables-nft-save v1.8.10 (nf_tables) on Sat Sep 28 13:01:01 2024
*filter
:INPUT ACCEPT [18477148:45351507050]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [12661494:8498414420]
-A INPUT -i mpqemubr0 -p tcp -m tcp --dport 53 -m comment --comment "generated for Multipass network mpqemubr0" -j ACCEPT
-A INPUT -i mpqemubr0 -p udp -m udp --dport 53 -m comment --comment "generated for Multipass network mpqemubr0" -j ACCEPT
-A INPUT -i mpqemubr0 -p udp -m udp --dport 67 -m comment --comment "generated for Multipass network mpqemubr0" -j ACCEPT
-A FORWARD -i mpqemubr0 -o mpqemubr0 -m comment --comment "generated for Multipass network mpqemubr0" -j ACCEPT
-A FORWARD -s 10.20.178.0/24 -i mpqemubr0 -m comment --comment "generated for Multipass network mpqemubr0" -j ACCEPT
-A FORWARD -d 10.20.178.0/24 -o mpqemubr0 -m conntrack --ctstate RELATED,ESTABLISHED -m comment --comment "generated for Multipass network mpqemubr0" -j ACCEPT
-A FORWARD -i mpqemubr0 -m comment --comment "generated for Multipass network mpqemubr0" -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -o mpqemubr0 -m comment --comment "generated for Multipass network mpqemubr0" -j REJECT --reject-with icmp-port-unreachable
-A OUTPUT -o mpqemubr0 -p tcp -m tcp --sport 53 -m comment --comment "generated for Multipass network mpqemubr0" -j ACCEPT
-A OUTPUT -o mpqemubr0 -p udp -m udp --sport 53 -m comment --comment "generated for Multipass network mpqemubr0" -j ACCEPT
-A OUTPUT -o mpqemubr0 -p udp -m udp --sport 67 -m comment --comment "generated for Multipass network mpqemubr0" -j ACCEPT
COMMIT
# Completed on Sat Sep 28 13:01:01 2024
# Generated by iptables-nft-save v1.8.10 (nf_tables) on Sat Sep 28 13:01:01 2024
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
levkropp commented 1 month ago

Hi everyone,

To continue troubleshooting this network issue, we're sharing an experimental build using a different C++ library, POCO::Net, to handle the connections instead of Qt::Network.

Please try this build and let us know your results—it should help us determine if the issue is indeed with Qt's network library or something else in the stack.

You can install the snap with the command sudo snap install multipass.snap --devmode --dangerous

https://send.bitwarden.com/#mo3xPOafnkWMSbH-AQSW3g/61VEDzK4N8keoibjr2OKFA

mcw-work commented 1 month ago

Hi, sorry for the delay in giving it a go. This one seems to work fine for me. Reverting back to the latest track from snap store breaks it again. Looks like whatever you did has found the problem.

Thanks for your efforts.

s-makin commented 1 month ago

Hi everyone,

To continue troubleshooting this network issue, we're sharing an experimental build using a different C++ library, POCO::Net, to handle the connections instead of Qt::Network.

Please try this build and let us know your results—it should help us determine if the issue is indeed with Qt's network library or something else in the stack.

You can install the snap with the command sudo snap install multipass.snap --devmode --dangerous

https://send.bitwarden.com/#mo3xPOafnkWMSbH-AQSW3g/61VEDzK4N8keoibjr2OKFA

I didn't get as far as trying this - I was going to, and tried running the multipass launch command first (just to test) and it actually worked first time. As mcw-work said, whatever you did behind the scenes seems to have fixed the problem for me as well. I've tried running the launch command several times, each with a different Ubuntu release, and they all worked perfectly.

Thanks for all your work on fixing this!

ricab commented 1 month ago

That's interesting. I am glad that it works for you now @s-makin, but we didn't do anything outside that experimental package.