Apple M1 / Hypervisor.framework support

[flying-sausages]

What are you trying to do? Run Multipass to launch instances on top of Apple Silicon devices

What's your proposed solution? Restrict multipass on AS to use the Virtualisation API intrdouced in Big Sur only and ensure ARM builds of ubuntu are being installed

Additional context I have 0 experience in any of the underlying technologies that are used here. I know 1.5.0 has Big Sur support on Intel through the API above, and so I'm wondering what are the extra steps that need to be taken care in order to allow users to have ARM-based VMs.

ED: Latest build

1.8.0 build 787

[adamcstephens]

The only other virtualization I have is lima, which I only installed after multipass stopped working for me.

[lethargosapatheia]

I tried vftool, but I'm probably doing something wrong:

$ ./vftool -k ../../../vm-test/ubuntu-20.04-server-cloudimg-arm64-vmlinuz-generic -i ../../../vm-test/ubuntu-20.04-server-cloudimg-arm64-initrd-generic ../../../vm-test/ubuntu-20.04-server-cloudimg-arm64.img -p 2 -m 4096 -a "console=hvc0 root=/dev/vda1"
2021-09-28 22:58:15.770 vftool[42158:751629] vftool (v0.3 10/12/2020) starting
2021-09-28 22:58:15.771 vftool[42158:751629] +++ kernel at ../../../vm-test/ubuntu-20.04-server-cloudimg-arm64-vmlinuz-generic, initrd at ../../../vm-test/ubuntu-20.04-server-cloudimg-arm64-initrd-generic, cmdline 'console=hvc0', 1 cpus, 512MB memory
2021-09-28 22:58:15.771 vftool[42158:751629] +++ fd 3 connected to /dev/ttys001
2021-09-28 22:58:15.771 vftool[42158:751629] +++ Waiting for connection to:  /dev/ttys001
2021-09-28 23:08:54.649 vftool[42158:751629] +++ Configuration validated.
2021-09-28 23:08:54.650 vftool[42158:751629] +++ canStart = 1, vm state 0
2021-09-28 23:08:54.696 vftool[42158:759748] --- VM start error: Error Domain=VZErrorDomain Code=1 "The virtual machine failed to start." UserInfo={NSLocalizedFailure=Internal Virtualization error., NSLocalizedFailureReason=The virtual machine failed to start.}

The last lines appear only after I execute screen to connect to the tty.

[townsend2010]

Hey all,

We've dropped the first Release Candidate for Apple M1 support. It can be found at: https://multipass-ci.s3.amazonaws.com/multipass-1.8.0-rc.849%2Bgb6625c10.mac-Darwin.pkg

This is a Universal package that supports both Arm and Intel Macs.

The Intel side of things will now support a QEMU driver, but is not enabled by default. In order to enable the QEMU driver, you need to do: $ sudo multipass set local.driver=qemu

Please note that if you currently have Hyperkit-based instances, those will not be available when using the QEMU driver, but they will be preserved and can be used again when setting the driver to hyperkit. We have plans to have a conversion utility to allow using the hyperkit instances with the qemu driver, but we aren't there yet :smile:

Lastly, there have a been a few reports during the beta testing that launching instances will suddenly start timing out where before it would work. There seems to be something going on with the Apple virtualized networking that we haven't been able to solve yet. We are tracking that in #2265 and if you run in to that, please report it there.

Thank you, good luck, and let us know how it goes!

[franz2256]

Hello,

I tested it and worked fine on a mac mini. Did you know if the bridge option to access the vm from an outside computer is implemented now ? thank you for your work

[townsend2010]

Hi @franz2256,

Thanks for trying it out and reporting your success.

Did you know if the bridge option to access the vm from an outside computer is implemented now ?

Not yet and won't be in the initial release, but we plan on making that available in later release.

[gluxon]

Hey @townsend2010, thanks a lot for the RC! I tested it out an an ARM iMac and everything worked well.

[bboykin87]

I was also able to get an instance running and open a shell successfully using the RC build that was posted here. Thanks!

M1 MBP

[PrernaWeb]

Can confirm I have multiple instances up and running using the RC build on an M1 MacBook Air. Nice work guys.

[louielin]

I can't mount folder at this version.

multipass   1.8.0-rc.849+gb6625c10.mac
multipassd  1.8.0-rc.849+gb6625c10.mac

mount failed: source "{Path}" is not readable

When I give the access right for multipassd by Settings -> Security & Privacy -> Privacy -> Full Disk Access and run multipass mount again.

It would still show the error message and auto cancel the access right in Settings -> Security & Privacy -> Privacy -> Full Disk Access

[tgerov]

I tested the latest RC and it works well on the MacBook Pro M1, GJ.

[m-bers]

Since this runs on QEMU and not hyperkit, any chance we could get image launching support for the Apple Silicon release?

1260

[Boes-man]

Hi @franz2256,

Thanks for trying it out and reporting your success.

Did you know if the bridge option to access the vm from an outside computer is implemented now ?

Not yet and won't be in the initial release, but we plan on making that available in later release.

Hello, can you please clarify. With this rc there is no network bridge and it is also not planned for the ga release? Its one of the key reasons I use multipass. Perhaps its only not availalble for these M1 architectures, but will continue for the current architectures (intel). Thanks.

[Saviq]

Hello, can you please clarify. With this rc there is no network bridge and it is also not planned for the ga release? Its one of the key reasons I use multipass. Perhaps its only not availalble for these M1 architectures, but will continue for the current architectures (intel). Thanks.

It's not there on Intel macOS today (unless you're using VirtualBox), either - but nothing changes there. The statement above was about Apple Silicon only.

We plan to close the gap in the coming months.

[Saviq]

This is released in v1.8.0 now!

[Boes-man]

Hello, can you please clarify. With this rc there is no network bridge and it is also not planned for the ga release? Its one of the key reasons I use multipass. Perhaps its only not availalble for these M1 architectures, but will continue for the current architectures (intel). Thanks.

It's not there on Intel macOS today (unless you're using VirtualBox), either - but nothing changes there. The statement above was about Apple Silicon only.

We plan to close the gap in the coming months.

Thanks @Saviq , fyi I use intel macOS + multipass only (no virtualbox). Multipass creates a network bridge to the VM and I can ssh/ping/etc straight to the VM's IP from my laptop. Super useful and a major reason I support multipass (and microk8s). Losing this feature would be a bummer :)

Congratulations on the 1.8.0 release!

[kenji21]

Take care that could use only a single IP (it is in multipass documentation), the origin is from vmnet

"The interface receives a private IPv4 address via DHCP. IPv4 traffic originating from the guest operating system must use the private IPv4 address. Packets sent from a different IPv4 address are dropped by the system."

I have asked on apple dev forums if there is a way to use multiple IPs : https://developer.apple.com/forums/thread/693146

[Saviq]

Thanks @Saviq , fyi I use intel macOS + multipass only (no virtualbox). Multipass creates a network bridge to the VM and I can ssh/ping/etc straight to the VM's IP from my laptop. Super useful and a major reason I support multipass (and microk8s). Losing this feature would be a bummer :)

Ah yes! That bridged interface :)

We will certainly keep that if at all possible. What we meant by "bridging" was additional networks that you can configure with --network … - but not (yet) on macOS :)

Congratulations on the 1.8.0 release!

Thanks! :)

[maberer]

Thanks @Saviq for your hard work!

So can we suggest a change to https://isapplesiliconready.com/app/Multipass ?

Is multipass a Universal app now?

[Saviq]

Is multipass a Universal app now?

It is, yeah!

[maberer]

Great. Submitted a change to https://isapplesiliconready.com/app/Multipass

[mlb5000]

@Saviq is there a place to track --networks support for non VirtualBox setups? As it stands, I believe multi-NIC support is impossible on Apple Silicon, blocking compatibility with my particular network setup for my K8S cluster.

Edit: actually, I believe this is a larger issue for me as a software engineer on the M1 platform as a whole. Without networks support, I don't think I could expose any resources launched by multipass on my local network, which would be a big limitation.

[Saviq]

@mlb5000 in progress, you should be able to try the package from here:

https://github.com/canonical/multipass/pull/2462/checks?check_run_id=5375689050

[mlb5000]

@Saviq oh dang, thanks! I will try to do that this week and report back.

[mlb5000]

@Saviq can confirm the Networks option is there now. Is there documentation somewhere for how to add a network/launch option to an existing instance? I'd like to either add --network bridged or a specific network interface to my existing microk8s-vm instance since I'm not sure I can have the MicroK8S installer pass the network options into multipass during the install process.

[mlb5000]

@Saviq ok. so can confirm the current 1.9 dev build isn't ready for this yet. Attempting to start a "simple" instance using

multipass launch --network en11 --name networktest

results in a timeout, the VM being set to an Unknown status, and appears to take down the multipassd service, as subsequent attempts cannot connect to the multipass socket.

I also noticed that my driver is set to qemu, and that it says invalid driver when I try change it to hyperkit.

[eminchen]

I having error after installing and launching multipass:

multipass start [2022-03-24T14:21:43.129] [error] [primary] process error occurred Crashed program: qemu-system-aarch64; error: Process crashed [2022-03-24T14:21:43.132] [error] [primary] error: program: qemu-system-aarch64; error: Process crashed start failed: cannot connect to the multipass socket
Please ensure multipassd is running and '/var/run/multipass_socket' is accessible

Any tips how to troubleshoot it?