Starting instance keeps loading cert and Infinite loop

[garfield-tr]

Describe the bug Multipass instance stucking in Starting status after rebooting Mac then starting the instance. From the log, it stuck on Verifying Cert.

To Reproduce How, and what happened?

1. Launch an instance
2. stop the instance, quit the multipass application
3. reboot
4. Launch multipass, then run multipass start {instance_name}.

Expected behavior The instance get started successfully, and I'm able to login the instance via ssh

Logs

023-08-17T12:33:53.352] [warning] [Qt] Empty filename passed to function
[2023-08-17T12:33:53.630] [trace] [client cert store] Loading client certs from /var/root/Library/Application Support/multipassd/authenticated-certs
[2023-08-17T12:33:55.122] [trace] [url downloader] Found https://multipass.run/static/latest-release.json in cache: false
[2023-08-17T12:33:55.122] [debug] [update] Latest Multipass release available is version 1.12.2
[2023-08-17T12:33:55.465] [trace] [url downloader] Found https://cloud-images.ubuntu.com/releases/streams/v1/index.json in cache: false
[2023-08-17T12:33:55.511] [trace] [url downloader] Found https://cloud-images.ubuntu.com/releases/streams/v1/com.ubuntu.cloud:released:download.json in cache: true
[2023-08-17T12:33:56.570] [trace] [url downloader] Found https://cloud-images.ubuntu.com/buildd/daily/streams/v1/index.json in cache: false
[2023-08-17T12:33:58.170] [trace] [url downloader] Found https://cloud-images.ubuntu.com/buildd/daily/streams/v1/com.ubuntu.cloud:daily:download.json in cache: false
[2023-08-17T12:33:58.185] [trace] [url downloader] Found https://cdimage.ubuntu.com/ubuntu-core/appliances/streams/v1/index.json in cache: true
[2023-08-17T12:33:58.186] [trace] [url downloader] Found https://cdimage.ubuntu.com/ubuntu-core/appliances/streams/v1/com.ubuntu.appliances:released:download.json in cache: true
[2023-08-17T12:33:58.187] [info] [VMImageHost] Did not find any supported products in "appliance"
[2023-08-17T12:33:58.187] [trace] [url downloader] Found https://codeload.github.com/canonical/multipass-blueprints/zip/refs/heads/main in cache: true
[2023-08-17T12:33:58.193] [debug] [blueprint provider] Loading "anbox-cloud-appliance" v1
[2023-08-17T12:33:58.194] [debug] [blueprint provider] Loading "charm-dev" v1
[2023-08-17T12:33:58.194] [debug] [blueprint provider] Loading "docker" v1
[2023-08-17T12:33:58.194] [debug] [blueprint provider] Loading "jellyfin" v1
[2023-08-17T12:33:58.195] [debug] [blueprint provider] Loading "minikube" v1
[2023-08-17T12:33:58.195] [debug] [blueprint provider] Loading "ros-noetic" v1
[2023-08-17T12:33:58.196] [debug] [blueprint provider] Loading "ros2-humble" v1
[2023-08-17T12:33:58.206] [info] [rpc] gRPC listening on unix:/var/run/multipass_socket
[2023-08-17T12:33:58.220] [debug] [qemu-img] [912] started: qemu-img snapshot -l /var/root/Library/Application Support/multipassd/qemu/vault/instances/yd/ubuntu-18.04-server-cloudimg-arm64.img
[2023-08-17T12:33:58.272] [info] [sshfs-mount-handler] initializing mount /Users/wuzehuai/Workspace/Trackrevenue/yd => /vagrant in 'yd'
[2023-08-17T12:33:58.273] [info] [sshfs-mount-handler] initializing mount /Users/wuzehuai/Workspace/Trackrevenue/multipass/shared_folder => /home/vagrant/sf in 'yd'
[2023-08-17T12:33:58.273] [info] [daemon] Starting Multipass 1.12.2+mac
[2023-08-17T12:33:58.273] [info] [daemon] Daemon arguments: /Library/Application Support/com.canonical.multipass/bin/multipassd --verbosity debug --verbosity trace
[2023-08-17T12:34:08.233] [trace] [client cert store] Verifying cert:
-----BEGIN CERTIFICATE-----
{{certificate string here}}
-----END CERTIFICATE-----

There're more logs with the below content being repeated every 1 sec, it seemed that multipass are keep verifying the cert file.

[2023-08-17T12:34:09.222] [trace] [client cert store] Verifying cert:
-----BEGIN CERTIFICATE-----
{{certificate string here}}
-----END CERTIFICATE-----

Additional info

• OS: macOS 13.5 M1Max
  • multipass version

    multipass   1.12.2+mac
    multipassd  1.12.2+mac

  • multipass info --all

    Name:           yd
    State:          Stopped
    IPv4:           --
    Release:        --
    Image hash:     9930a57427ad (Ubuntu 18.04 LTS)
    CPU(s):         --
    Load:           --
    Disk usage:     --
    Memory usage:   --
    Mounts:          /{my home path}/multipass/shared_folder => /home/vagrant/sf
              UID map: 501:default
              GID map: 20:default
              /{my home path}/workspace                     => /vagrant
              UID map: 501:default
              GID map: 20:default

  • multipass get local.driver

    qemu

    Additional context Please let me know if there's any information needed. Thanks

[luis4a0]

Hi @garfield-tr! I think certificate verification isn't related to this issue. On the other hand, are you using some VPN software? Are you using the macOS firewall? Thanks!

[garfield-tr]

Hey, @luis4a0 thanks for replying so quickly. No, I didn't use a VPN, but I do use proxy software that uses Socks5 or VMESS protocols. I tried to turn off the macOS firewall, the issue still exists.

[luis4a0]

Hey @garfield-tr! Would it be possible for you to try disabling your proxy software? That way, we can determine if the problem comes from there. Thanks!

[garfield-tr]

Yeah, I tried several ways to see if the issue still exists, I tried the following things,

1. stop the proxy and reset network settings: not work
2. restarting multipass and rebooting the Mac: not work
3. a fresh new instance still works for me

[luis4a0]

Hi @garfield-tr, so it seems the network is not the issue. Maybe your old instances got somehow corrupted. Can you please run qemu-img check /var/root/Library/Application Support/multipassd/qemu/vault/instances/yd/ubuntu-18.04-server-cloudimg-arm64.img, after making sure the instance is stopped? To stop all instances, just run multipass stop --all, and then sudo killall -9 qemu-system-aarch64.

Thanks!

[garfield-tr]

OK, Thanks @luis4a0 , I'll give a try

[chafey]

I am having the same issue - any ideas on how to fix?

[townsend2010]

Hi @chafey!

Which issue in here are you seeing? The loading of the cert message every second? If that is the case, this is normal. It's from the tray icon client that polls every second.

[chafey]

The issue i am having is that instances fail to start. I enabled trace debugging and noticed the cert message every second and google lead me here so I was hoping it was causing it. Good to know its not the cause, unfortunately it is still broken even after rebooting, reinstalling, etc

[townsend2010]

Hi @chafey,

Ok, I see. I think it would be better if you could just open a new issue and provide the info and logs that the template asks for.