search

canonical / multipass

Multipass orchestrates virtual Ubuntu instances
https://multipass.run
GNU General Public License v3.0
7.56k stars 636 forks source link

Multipass Network issue #3570

Open Ash-Garg opened 2 weeks ago

Ash-Garg commented 2 weeks ago

I need help with Multipass as my VM has stopped working. It appears to be a networking issue. I have managed macOS ( sonoma 14.5 ) with the firewall enabled. Multipassd is explicitly allowed, and “Block all incoming connections” is disabled.

I have followed all the steps outlined in this guide: Troubleshoot Networking on macOS.

Additionally, I tried to create a new VM, but I encountered the same issue (the machine remains in an unknown status; either SSH is not possible or the machine is not getting an IP).

I enabled tcpdump on the bridge100 interface to investigate and found that the new VM was sending BOOTP/DHCP requests but not receiving any replies. After multiple unload/load cycles of Bootpd, I eventually disabled Bootpd and installed another DHCP server, isc-dhcp (instructions in the link below).

https://github.com/lima-vm/lima/issues/1259

The new VM then received an IP address, and I got a popup in macOS to allow incoming connections to isc-dhcp. However, even after that, Multipassd was not able to move the VM to the running state (SSH still seemed to be an issue).

Regarding my existing VM, there is never a DHCP request. With the old .64 subnet, the VM was getting an IP right away (I believe) as there was an ARP request from 64.1.

22:37:12.740053 ARP, Request who-has 192.168.64.5 tell 192.168.64.1, length 28 22:37:13.290653 IP 192.168.64.1 > igmp.mcast.net: igmp v3 report, 1 group record(s) 22:37:13.499397 IP 192.168.64.1.mdns > mdns.mcast.net.mdns: 0 [1au] PTR (QM)? d.d.0.3.2.e.e.f.f.f.0.0.4.5.0.5.0.0.0.0.0.0.0.0.0.0.0.0.0.8.e.f.ip6.arpa. (119) 22:37:13.499464 IP6 fe80::f4d4:88ff:fe57:d364.mdns > ff02::fb.mdns: 0 [1au] PTR (QM)? d.d.0.3.2.e.e.f.f.f.0.0.4.5.0.5.0.0.0.0.0.0.0.0.0.0.0.0.0.8.e.f.ip6.arpa. (119) 22:37:13.740846 ARP, Request who-has 192.168.64.5 tell 192.168.64.1, length 28

However, since I changed the subnet to .65, there is no IP and no DHCP requests either. I am unsure what to do next.

ricab commented 2 weeks ago

Hi @Ash-Garg, thanks for moving the post in here.

It looks like you're stumbling on the dreaded firewall issue: #2387. This appears to be a bug somewhere in Apple's firewall (or possibly vmnet/bootp/launchd or their configuration), which blocks bootp from giving out IPs. As you found, this affects other clients of the vmnet API.

If you have admin status, you can try to either disable the firewall entirely, or try the workaround in that issue:

/usr/libexec/ApplicationFirewall/socketfilterfw --add /usr/libexec/bootpd
/usr/libexec/ApplicationFirewall/socketfilterfw --unblock /usr/libexec/bootpd

Some people report success with the socketfilterfw commands. Some report that they need to run this after every reboot.

isc-dhcp is discontinued. Kea is the replacement, with "best-effort" support on macOS. In any case, the vmnet API says:

The interface receives a private IPv4 address via DHCP. IPv4 traffic originating from the guest operating system must use the private IPv4 address. Packets sent from a different IPv4 address are dropped by the system.

So I wouldn't have high hopes for an alternative DHCP approach with vmnet. Also, Multipass wouldn't know about alternative IPs anyway: we get them from /var/db/dhcpd_leases, which Apple's implementation populates (or is supposed to).

Ash-Garg commented 2 weeks ago

Thanks for the response @ricab , I do not have Admin access to macbook. Firewall is managed by Jamf Policy. What will be the workaround as I didn't find any in the mentioned thread.

ricab commented 2 weeks ago

Hi @Ash-Garg, I am afraid that there is no other workaround that we know of, apart from switching to the VirtualBox backend, if you still happen to be on Intel hardware: multipass set local.driver=virtualbox.

I really wish that there was something more that I could give you and all other users in your situation :frowning_face: This has been reported to Apple, but unfortunately they aren't doing anything about it. I have suggested other users knock on their door and alert them to the problem. Their API just doesn't work! The only other thing we could do would be to switch from vmnet to some sort of userspace network stack. That is how VirtualBox does it, but it comes with its own drawbacks and I am not sure if QEMU could work with it.

Ash-Garg commented 1 week ago

Hi @ricab Thanks for your help so far. I've managed to work with the support team to solve the firewall issue by manually adding the necessary rules to the macOS firewall. This has resolved the issue for the new instance I created in Multipass.

However, I'm still facing an issue with my existing instance, which contains all my data and configurations. Below are the tcpdump logs. As you can see, the instance is not even sending a DHCP request. This wasn't the case before, as the instance used to get a 64.5 address, and I would see ARP requests in the logs. This problem started when I changed the subnet to 65.1 during troubleshooting.

I was hoping that solving the firewall issue would allow the instance to get an IP address again, but it’s still not working. I have even reverted the configuration back to the 64.1 subnet, but with no luck. How should I proceed to resolve this issue?

tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on bridge100, link-type EN10MB (Ethernet), snapshot length 524288 bytes 19:53:44.431285 IP 192.168.64.1.mdns > mdns.mcast.net.mdns: 0 [16q] [1au] PTR (QM)? lb._dns-sd._udp.local. PTR (QM)? _airport._tcp.local. PTR (QM)? _rdlink._tcp.local. PTR (QM)? _uscan._tcp.local. PTR (QM)? _ippusb._tcp.local. PTR (QM)? _scanner._tcp.local. PTR (QM)? _ptp._tcp.local. PTR (QM)? _uscans._tcp.local. PTR (QM)? _pdl-datastream._tcp.local. PTR (QM)? _ipps._tcp.local. PTR (QM)? _ipp._tcp.local. PTR (QM)? _printer._tcp.local. PTR (QM)? _apple-mobdev._tcp.local. PTR (QM)? 8e2707a5._sub._apple-mobdev2._tcp.local. PTR (QM)? _apple-pairable._tcp.local. PTR (QM)? _companion-link._tcp.local. (328) 19:53:44.431388 IP6 fe80::f4d4:88ff:fe57:d364.mdns > ff02::fb.mdns: 0 [16q] [1au] PTR (QM)? lb._dns-sd._udp.local. PTR (QM)? _airport._tcp.local. PTR (QM)? _rdlink._tcp.local. PTR (QM)? _uscan._tcp.local. PTR (QM)? _ippusb._tcp.local. PTR (QM)? _scanner._tcp.local. PTR (QM)? _ptp._tcp.local. PTR (QM)? _uscans._tcp.local. PTR (QM)? _pdl-datastream._tcp.local. PTR (QM)? _ipps._tcp.local. PTR (QM)? _ipp._tcp.local. PTR (QM)? _printer._tcp.local. PTR (QM)? _apple-mobdev._tcp.local. PTR (QM)? 8e2707a5._sub._apple-mobdev2._tcp.local. PTR (QM)? _apple-pairable._tcp.local. PTR (QM)? _companion-link._tcp.local. (328) 19:53:44.527312 IP6 fe80::f4d4:88ff:fe57:d364 > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 19:53:44.593526 IP 192.168.64.1.mdns > mdns.mcast.net.mdns: 0 [1au] PTR (QU)? 4.6.3.d.7.5.e.f.f.f.8.8.4.d.4.f.0.0.0.0.0.0.0.0.0.0.0.0.0.8.e.f.ip6.arpa. (119) 19:53:44.593622 IP6 fe80::f4d4:88ff:fe57:d364.mdns > ff02::fb.mdns: 0 [1au] PTR (QU)? 4.6.3.d.7.5.e.f.f.f.8.8.4.d.4.f.0.0.0.0.0.0.0.0.0.0.0.0.0.8.e.f.ip6.arpa. (119) 19:53:45.412534 IP 192.168.64.1.mdns > mdns.mcast.net.mdns: 0 [1au] PTR (QM)? 4.6.3.d.7.5.e.f.f.f.8.8.4.d.4.f.0.0.0.0.0.0.0.0.0.0.0.0.0.8.e.f.ip6.arpa. (119) 19:53:45.412734 IP6 fe80::f4d4:88ff:fe57:d364.mdns > ff02::fb.mdns: 0 [1au] PTR (QM)? 4.6.3.d.7.5.e.f.f.f.8.8.4.d.4.f.0.0.0.0.0.0.0.0.0.0.0.0.0.8.e.f.ip6.arpa. (119) 19:53:46.205370 IP 192.168.64.1 > igmp.mcast.net: igmp v3 report, 1 group record(s) 19:53:47.438060 IP 192.168.64.1.mdns > mdns.mcast.net.mdns: 0 [17q] [1au] PTR (QM)? lb._dns-sd._udp.local. PTR (QM)? _airport._tcp.local. PTR (QM)? _rdlink._tcp.local. PTR (QM)? _uscan._tcp.local. PTR (QM)? _ippusb._tcp.local. PTR (QM)? _scanner._tcp.local. PTR (QM)? _ptp._tcp.local. PTR (QM)? _uscans._tcp.local. PTR (QM)? _pdl-datastream._tcp.local. PTR (QM)? _ipps._tcp.local. PTR (QM)? _ipp._tcp.local. PTR (QM)? _printer._tcp.local. PTR (QM)? _apple-mobdev._tcp.local. PTR (QM)? 8e2707a5._sub._apple-mobdev2._tcp.local. PTR (QM)? _apple-pairable._tcp.local. PTR (QM)? _companion-link._tcp.local. PTR (QM)? 4.6.3.d.7.5.e.f.f.f.8.8.4.d.4.f.0.0.0.0.0.0.0.0.0.0.0.0.0.8.e.f.ip6.arpa. (406) 19:53:47.438145 IP6 fe80::f4d4:88ff:fe57:d364.mdns > ff02::fb.mdns: 0 [17q] [1au] PTR (QM)? lb._dns-sd._udp.local. PTR (QM)? _airport._tcp.local. PTR (QM)? _rdlink._tcp.local. PTR (QM)? _uscan._tcp.local. PTR (QM)? _ippusb._tcp.local. PTR (QM)? _scanner._tcp.local. PTR (QM)? _ptp._tcp.local. PTR (QM)? _uscans._tcp.local. PTR (QM)? _pdl-datastream._tcp.local. PTR (QM)? _ipps._tcp.local. PTR (QM)? _ipp._tcp.local. PTR (QM)? _printer._tcp.local. PTR (QM)? _apple-mobdev._tcp.local. PTR (QM)? 8e2707a5._sub._apple-mobdev2._tcp.local. PTR (QM)? _apple-pairable._tcp.local. PTR (QM)? _companion-link._tcp.local. PTR (QM)? 4.6.3.d.7.5.e.f.f.f.8.8.4.d.4.f.0.0.0.0.0.0.0.0.0.0.0.0.0.8.e.f.ip6.arpa. (406) 19:53:56.457113 IP 192.168.64.1.mdns > mdns.mcast.net.mdns: 0 [16q] [1au] PTR (QM)? lb._dns-sd._udp.local. PTR (QM)? _airport._tcp.local. PTR (QM)? _rdlink._tcp.local. PTR (QM)? _uscan._tcp.local. PTR (QM)? _ippusb._tcp.local. PTR (QM)? _scanner._tcp.local. PTR (QM)? _ptp._tcp.local. PTR (QM)? _uscans._tcp.local. PTR (QM)? _pdl-datastream._tcp.local. PTR (QM)? _ipps._tcp.local. PTR (QM)? _ipp._tcp.local. PTR (QM)? _printer._tcp.local. PTR (QM)? _apple-mobdev._tcp.local. PTR (QM)? 8e2707a5._sub._apple-mobdev2._tcp.local. PTR (QM)? _apple-pairable._tcp.local. PTR (QM)? _companion-link._tcp.local. (328) 19:53:56.457226 IP6 fe80::f4d4:88ff:fe57:d364.mdns > ff02::fb.mdns: 0 [16q] [1au] PTR (QM)? lb._dns-sd._udp.local. PTR (QM)? _airport._tcp.local. PTR (QM)? _rdlink._tcp.local. PTR (QM)? _uscan._tcp.local. PTR (QM)? _ippusb._tcp.local. PTR (QM)? _scanner._tcp.local. PTR (QM)? _ptp._tcp.local. PTR (QM)? _uscans._tcp.local. PTR (QM)? _pdl-datastream._tcp.local. PTR (QM)? _ipps._tcp.local. PTR (QM)? _ipp._tcp.local. PTR (QM)? _printer._tcp.local. PTR (QM)? _apple-mobdev._tcp.local. PTR (QM)? 8e2707a5._sub._apple-mobdev2._tcp.local. PTR (QM)? _apple-pairable._tcp.local. PTR (QM)? _companion-link._tcp.local. (328) 19:53:57.391770 IP6 fe80::f4d4:88ff:fe57:d364 > ff02::1: ICMP6, router advertisement, length 88 19:54:13.466140 IP6 fe80::f4d4:88ff:fe57:d364 > ff02::1: ICMP6, router advertisement, length 88 19:54:23.511462 IP 192.168.64.1.mdns > mdns.mcast.net.mdns: 0 [16q] [1au] PTR (QM)? lb._dns-sd._udp.local. PTR (QM)? _airport._tcp.local. PTR (QM)? _rdlink._tcp.local. PTR (QM)? _uscan._tcp.local. PTR (QM)? _ippusb._tcp.local. PTR (QM)? _scanner._tcp.local. PTR (QM)? _ptp._tcp.local. PTR (QM)? _uscans._tcp.local. PTR (QM)? _pdl-datastream._tcp.local. PTR (QM)? _ipps._tcp.local. PTR (QM)? _ipp._tcp.local. PTR (QM)? _printer._tcp.local. PTR (QM)? _apple-mobdev._tcp.local. PTR (QM)? 8e2707a5._sub._apple-mobdev2._tcp.local. PTR (QM)? _apple-pairable._tcp.local. PTR (QM)? _companion-link._tcp.local. (328) 19:54:23.511646 IP6 fe80::f4d4:88ff:fe57:d364.mdns > ff02::fb.mdns: 0 [16q] [1au] PTR (QM)? lb._dns-sd._udp.local. PTR (QM)? _airport._tcp.local. PTR (QM)? _rdlink._tcp.local. PTR (QM)? _uscan._tcp.local. PTR (QM)? _ippusb._tcp.local. PTR (QM)? _scanner._tcp.local. PTR (QM)? _ptp._tcp.local. PTR (QM)? _uscans._tcp.local. PTR (QM)? _pdl-datastream._tcp.local. PTR (QM)? _ipps._tcp.local. PTR (QM)? _ipp._tcp.local. PTR (QM)? _printer._tcp.local. PTR (QM)? _apple-mobdev._tcp.local. PTR (QM)? 8e2707a5._sub._apple-mobdev2._tcp.local. PTR (QM)? _apple-pairable._tcp.local. PTR (QM)? _companion-link._tcp.local. (328) 19:55:06.575615 IP 192.168.64.1.mdns > mdns.mcast.net.mdns: 0 [16q] PTR (QM)? lb._dns-sd._udp.local. PTR (QM)? _airport._tcp.local. PTR (QM)? _rdlink._tcp.local. PTR (QM)? _uscan._tcp.local. PTR (QM)? _ippusb._tcp.local. PTR (QM)? _scanner._tcp.local. PTR (QM)? _ptp._tcp.local. PTR (QM)? _uscans._tcp.local. PTR (QM)? _pdl-datastream._tcp.local. PTR (QM)? _ipps._tcp.local. PTR (QM)? _ipp._tcp.local. PTR (QM)? _printer._tcp.local. PTR (QM)? _apple-mobdev._tcp.local. PTR (QM)? 8e2707a5._sub._apple-mobdev2._tcp.local. PTR (QM)? _apple-pairable._tcp.local. PTR (QM)? _companion-link._tcp.local. (299) 19:55:06.575770 IP 192.168.64.1.mdns > mdns.mcast.net.mdns: 0 [17q] PTR (QM)? lb._dns-sd._udp.local. PTR (QM)? _airport._tcp.local. PTR (QM)? _rdlink._tcp.local. PTR (QM)? _uscan._tcp.local. PTR (QM)? _ippusb._tcp.local. PTR (QM)? _scanner._tcp.local. PTR (QM)? _ptp._tcp.local. PTR (QM)? _uscans._tcp.local. PTR (QM)? _pdl-datastream._tcp.local. PTR (QM)? _ipps._tcp.local. PTR (QM)? _ipp._tcp.local. PTR (QM)? _printer._tcp.local. PTR (QM)? _apple-mobdev._tcp.local. PTR (QM)? 8e2707a5._sub._apple-mobdev2._tcp.local. PTR (QM)? _apple-pairable._tcp.local. PTR (QM)? _companion-link._tcp.local. PTR (QM)? _sleep-proxy._udp.local. (318) 19:55:06.575811 IP 192.168.64.1.mdns > mdns.mcast.net.mdns: 0 [16q] PTR (QM)? lb._dns-sd._udp.local. PTR (QM)? _airport._tcp.local. PTR (QM)? _rdlink._tcp.local. PTR (QM)? _uscan._tcp.local. PTR (QM)? _ippusb._tcp.local. PTR (QM)? _scanner._tcp.local. PTR (QM)? _ptp._tcp.local. PTR (QM)? _uscans._tcp.local. PTR (QM)? _pdl-datastream._tcp.local. PTR (QM)? _ipps._tcp.local. PTR (QM)? _ipp._tcp.local. PTR (QM)? _printer._tcp.local. PTR (QM)? _apple-mobdev._tcp.local. PTR (QM)? 8e2707a5._sub._apple-mobdev2._tcp.local. PTR (QM)? _apple-pairable._tcp.local. PTR (QM)? _companion-link._tcp.local. (299) 19:55:06.576088 IP6 fe80::f4d4:88ff:fe57:d364.mdns > ff02::fb.mdns: 0 [16q] PTR (QM)? lb._dns-sd._udp.local. PTR (QM)? _airport._tcp.local. PTR (QM)? _rdlink._tcp.local. PTR (QM)? _uscan._tcp.local. PTR (QM)? _ippusb._tcp.local. PTR (QM)? _scanner._tcp.local. PTR (QM)? _ptp._tcp.local. PTR (QM)? _uscans._tcp.local. PTR (QM)? _pdl-datastream._tcp.local. PTR (QM)? _ipps._tcp.local. PTR (QM)? _ipp._tcp.local. PTR (QM)? _printer._tcp.local. PTR (QM)? _apple-mobdev._tcp.local. PTR (QM)? 8e2707a5._sub._apple-mobdev2._tcp.local. PTR (QM)? _apple-pairable._tcp.local. PTR (QM)? _companion-link._tcp.local. (299) 19:55:06.576125 IP6 fe80::f4d4:88ff:fe57:d364.mdns > ff02::fb.mdns: 0 [17q] PTR (QM)? lb._dns-sd._udp.local. PTR (QM)? _airport._tcp.local. PTR (QM)? _rdlink._tcp.local. PTR (QM)? _uscan._tcp.local. PTR (QM)? _ippusb._tcp.local. PTR (QM)? _scanner._tcp.local. PTR (QM)? _ptp._tcp.local. PTR (QM)? _uscans._tcp.local. PTR (QM)? _pdl-datastream._tcp.local. PTR (QM)? _ipps._tcp.local. PTR (QM)? _ipp._tcp.local. PTR (QM)? _printer._tcp.local. PTR (QM)? _apple-mobdev._tcp.local. PTR (QM)? 8e2707a5._sub._apple-mobdev2._tcp.local. PTR (QM)? _apple-pairable._tcp.local. PTR (QM)? _companion-link._tcp.local. PTR (QM)? _sleep-proxy._udp.local. (318) 19:55:06.576160 IP6 fe80::f4d4:88ff:fe57:d364.mdns > ff02::fb.mdns: 0 [16q] PTR (QM)? lb._dns-sd._udp.local. PTR (QM)? _airport._tcp.local. PTR (QM)? _rdlink._tcp.local. PTR (QM)? _uscan._tcp.local. PTR (QM)? _ippusb._tcp.local. PTR (QM)? _scanner._tcp.local. PTR (QM)? _ptp._tcp.local. PTR (QM)? _uscans._tcp.local. PTR (QM)? _pdl-datastream._tcp.local. PTR (QM)? _ipps._tcp.local. PTR (QM)? _ipp._tcp.local. PTR (QM)? _printer._tcp.local. PTR (QM)? _apple-mobdev._tcp.local. PTR (QM)? 8e2707a5._sub._apple-mobdev2._tcp.local. PTR (QM)? _apple-pairable._tcp.local. PTR (QM)? _companion-link._tcp.local. (299) 19:55:06.576319 IP6 fe80::f4d4:88ff:fe57:d364.mdns > ff02::fb.mdns: 0 [16q] PTR (QM)? lb._dns-sd._udp.local. PTR (QM)? _airport._tcp.local. PTR (QM)? _rdlink._tcp.local. PTR (QM)? _uscan._tcp.local. PTR (QM)? _ippusb._tcp.local. PTR (QM)? _scanner._tcp.local. PTR (QM)? _ptp._tcp.local. PTR (QM)? _uscans._tcp.local. PTR (QM)? _pdl-datastream._tcp.local. PTR (QM)? _ipps._tcp.local. PTR (QM)? _ipp._tcp.local. PTR (QM)? _printer._tcp.local. PTR (QM)? _apple-mobdev._tcp.local. PTR (QM)? 8e2707a5._sub._apple-mobdev2._tcp.local. PTR (QM)? _apple-pairable._tcp.local. PTR (QM)? _companion-link._tcp.local. (299) 19:55:41.219531 IP 192.168.64.1.mdns > mdns.mcast.net.mdns: 0 PTR (QU)? _googlecast._tcp.local. (40) 19:55:41.219556 IP 192.168.64.1.mdns > mdns.mcast.net.mdns: 0 PTR (QU)? _googlecast._tcp.local. (40) 19:55:41.219567 IP 192.168.64.1.mdns > mdns.mcast.net.mdns: 0 PTR (QU)? _googlecast._tcp.local. (40) 19:55:41.219758 IP6 fe80::f4d4:88ff:fe57:d364.mdns > ff02::fb.mdns: 0 PTR (QU)? _googlecast._tcp.local. (40) 19:55:41.219773 IP6 fe80::f4d4:88ff:fe57:d364.mdns > ff02::fb.mdns: 0 PTR (QU)? _googlecast._tcp.local. (40) 19:55:41.219784 IP6 fe80::f4d4:88ff:fe57:d364.mdns > ff02::fb.mdns: 0 PTR (QU)? _googlecast._tcp.local. (40) 19:55:41.219795 IP6 fe80::f4d4:88ff:fe57:d364.mdns > ff02::fb.mdns: 0 PTR (QU)? _googlecast._tcp.local. (40) 19:55:42.218600 IP 192.168.64.1.mdns > mdns.mcast.net.mdns: 0 PTR (QM)? _googlecast._tcp.local. (40) 19:55:42.218690 IP6 fe80::f4d4:88ff:fe57:d364.mdns > ff02::fb.mdns: 0 PTR (QM)? _googlecast._tcp.local. (40) 19:55:45.225431 IP 192.168.64.1.mdns > mdns.mcast.net.mdns: 0 PTR (QM)? _googlecast._tcp.local. (40) 19:55:45.225538 IP6 fe80::f4d4:88ff:fe57:d364.mdns > ff02::fb.mdns: 0 PTR (QM)? _googlecast._tcp.local. (40) 19:55:54.243606 IP 192.168.64.1.mdns > mdns.mcast.net.mdns: 0 PTR (QM)? _googlecast._tcp.local. (40) 19:55:54.243654 IP6 fe80::f4d4:88ff:fe57:d364.mdns > ff02::fb.mdns: 0 PTR (QM)? _googlecast._tcp.local. (40) 19:55:54.541097 IP6 fe80::f4d4:88ff:fe57:d364 > ff02::1: ICMP6, router advertisement, length 88 19:56:21.299041 IP 192.168.64.1.mdns > mdns.mcast.net.mdns: 0 PTR (QM)? _googlecast._tcp.local. (40) 19:56:21.299104 IP6 fe80::f4d4:88ff:fe57:d364.mdns > ff02::fb.mdns: 0 PTR (QM)? _googlecast._tcp.local. (40) 19:56:55.616656 IP 192.168.64.1.mdns > mdns.mcast.net.mdns: 0*- [0q] 1/0/1 (Cache flush) A 192.168.2.16 (129) 19:56:55.616779 IP6 fe80::f4d4:88ff:fe57:d364.mdns > ff02::fb.mdns: 0

ricab commented 1 week ago

Hi @Ash-Garg, I wonder if that instance had to be killed and got corrupted for some reason. There could also be outdated leases. Can you please share the logs when you try to start the faulty instance?

I am glad you managed to get those firewall rules in! Hope they keep working for you throughout reboots. Please let us know how it goes.

Ash-Garg commented 1 week ago

Hi @ricab ,

Below is the logs from multipass when I try to start the faulty instance, I also tried to repair the image using qemu-img utility , It did repair the image but still no luck.

sudo /Library/Application\ Support/com.canonical.multipass/bin/qemu-img check -r all "/var/root/Library/Application Support/multipassd/qemu/vault/instances/superior-chihuahua/ubuntu-22.04-server-cloudimg-arm64.img"

Logs :

[2024-07-17T13:31:49.896] [debug] [superior-chihuahua] process working dir ''
[2024-07-17T13:31:49.896] [info] [superior-chihuahua] process program 'qemu-system-aarch64'
[2024-07-17T13:31:49.896] [info] [superior-chihuahua] process arguments '-machine, virt,gic-version=3, -accel, hvf, -drive, file=/Library/Application Support/com.canonical.multipass/bin/../Resources/qemu/edk2-aarch64-code.fd,if=pflash,format=raw,readonly=on, -cpu, host, -nic, vmnet-shared,model=virtio-net-pci,mac=52:54:00:e2:30:dd, -device, virtio-scsi-pci,id=scsi0, -drive, file=/var/root/Library/Application Support/multipassd/qemu/vault/instances/superior-chihuahua/ubuntu-22.04-server-cloudimg-arm64.img,if=none,format=qcow2,discard=unmap,id=hda, -device, scsi-hd,drive=hda,bus=scsi0.0, -smp, 2, -m, 4096M, -qmp, stdio, -chardev, null,id=char0, -serial, chardev:char0, -nographic, -cdrom, /var/root/Library/Application Support/multipassd/qemu/vault/instances/superior-chihuahua/cloud-init-config.iso'
[2024-07-17T13:31:49.904] [debug] [qemu-system-aarch64] [97914] started: qemu-system-aarch64 -machine virt,gic-version=3 -nographic -dump-vmstate /private/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/multipassd.lgRcqU
[2024-07-17T13:31:50.374] [info] [superior-chihuahua] process state changed to Starting
[2024-07-17T13:31:50.377] [info] [superior-chihuahua] process state changed to Running
[2024-07-17T13:31:50.377] [debug] [qemu-system-aarch64] [97915] started: qemu-system-aarch64 -machine virt,gic-version=3 -accel hvf -drive file=/Library/Application Support/com.canonical.multipass/bin/../Resources/qemu/edk2-aarch64-code.fd,if=pflash,format=raw,readonly=on -cpu host -nic vmnet-shared,model=virtio-net-pci,mac=52:54:00:e2:30:dd -device virtio-scsi-pci,id=scsi0 -drive file=/var/root/Library/Application Support/multipassd/qemu/vault/instances/superior-chihuahua/ubuntu-22.04-server-cloudimg-arm64.img,if=none,format=qcow2,discard=unmap,id=hda -device scsi-hd,drive=hda,bus=scsi0.0 -smp 2 -m 4096M -qmp stdio -chardev null,id=char0 -serial chardev:char0 -nographic -cdrom /var/root/Library/Application Support/multipassd/qemu/vault/instances/superior-chihuahua/cloud-init-config.iso
[2024-07-17T13:31:50.377] [info] [superior-chihuahua] process started
[2024-07-17T13:31:50.379] [debug] [superior-chihuahua] Waiting for SSH to be up
[2024-07-17T13:31:51.776] [debug] [superior-chihuahua] QMP: {"QMP": {"version": {"qemu": {"micro": 1, "minor": 2, "major": 8}, "package": ""}, "capabilities": ["oob"]}}

[2024-07-17T13:31:51.906] [debug] [superior-chihuahua] QMP: {"return": {}}
ricab commented 1 week ago

Hi @Ash-Garg, unfortunately I don't see the message that indicates that the network card is up inside the instance (a QMP message with NIC_RX_FILTER_CHANGED). That is an indication that something is going wrong with the boot, probably due to some corruption. You can try to run the instance manually and observe the boot process to confirm this with something like:

qemu-system-aarch64 -machine virt,gic-version=3 -accel hvf -drive file=/Library/Application\ Support/com.canonical.multipass/bin/../Resources/qemu/edk2-aarch64-code.fd,if=pflash,format=raw,readonly=on -cpu host -nic vmnet-shared,model=virtio-net-pci,mac=52:54:00:e2:30:dd -device virtio-scsi-pci,id=scsi0 -drive file=/var/root/Library/Application\ Support/multipassd/qemu/vault/instances/superior-chihuahua/ubuntu-22.04-server-cloudimg-arm64.img,if=none,format=qcow2,discard=unmap,id=hda -device scsi-hd,drive=hda,bus=scsi0.0 -smp 2 -m 4096M -qmp stdio -cdrom /var/root/Library/Application\ Support/multipassd/qemu/vault/instances/superior-chihuahua/cloud-init-config.iso

If that is indeed the case, the only option to try to recover your data would be to to mount the image with something like qemu-nbd. Here are some instructions. Search the web for "mount qcow2 image" for more. You can find the image in /var/root/Library/Application\ Support/multipassd/qemu/vault/instances/superior-chihuahua/ubuntu-22.04-server-cloudimg-arm64.img