qemu-system-aarch64: Memory exceeds the limit

[mknet]

Hello everybody,

while trying to start a vagrant instance I get the following error message:

==> default: Starting the instance...
A command executed by Vagrant didn't complete successfully!
The command run along with the output from the command is shown
below.

Command: ["qemu-system-aarch64", "-machine", "virt,accel=hvf,highmem=off", "-cpu", "cortex-a72", "-smp", "2", "-m", "4G", "-device", "virtio-net-device,netdev=net0", "-netdev", "user,id=net0,hostfwd=tcp::50022-:22", "-drive", "if=virtio,format=qcow2,file=/Users/marcelmrkoch/Development/mk/vagrant/.vagrant/machines/default/qemu/SAMv-queNHM/linked-box.img", "-drive", "if=pflash,format=raw,file=/Users/marcelmrkoch/Development/mk/vagrant/.vagrant/machines/default/qemu/SAMv-queNHM/edk2-aarch64-code.fd,readonly=on", "-drive", "if=pflash,format=raw,file=/Users/marcelmrkoch/Development/mk/vagrant/.vagrant/machines/default/qemu/SAMv-queNHM/edk2-arm-vars.fd", "-chardev", "socket,id=mon0,path=/Users/marcelmrkoch/.vagrant.d/tmp/vagrant-qemu/SAMv-queNHM/qemu_socket,server=on,wait=off", "-mon", "chardev=mon0,mode=readline", "-chardev", "socket,id=ser0,path=/Users/marcelmrkoch/.vagrant.d/tmp/vagrant-qemu/SAMv-queNHM/qemu_socket_serial,server=on,wait=off", "-serial", "chardev:ser0", "-pidfile", "/Users/marcelmrkoch/Development/mk/vagrant/.vagrant/machines/default/qemu/SAMv-queNHM/qemu.pid", "-parallel", "null", "-monitor", "none", "-display", "none", "-vga", "none", "-daemonize", {:notify=>[:stdout, :stderr, :stdin]}]

Stderr: qemu-system-aarch64: Addressing limited to 32 bits, but memory exceeds it by 1073741824 bytes

Stdout: 

Does anybody has a hint what I'm doing wrong?

Thank you

[ppggff]

Which version of qemu (run qemu-system-aarch64 --version to get), macos do you use?

I found this issue may be useful for your problem. The solution may: Try to give it 3GB instead of 4GB. Or use macos >= 12.4 and drop highmem=off and give it as much memory as you like. (you can config them through this)

[mknet]

% qemu-system-aarch64 --version
QEMU emulator version 7.0.0
Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers

I use macOS 12.5.

Thanks for you hints. I do my very best to solve it using em.

[mknet]

I tried it with

  config.vm.provider "qemu" do |qe|
    qe.memory = "8G"
    qe.machine = "virt,accel=hvf,highmem=on"
  end

and got

==> default: Starting the instance...
A command executed by Vagrant didn't complete successfully!
The command run along with the output from the command is shown
below.

Command: ["qemu-system-aarch64", "-machine", "virt,accel=hvf,highmem=on", "-cpu", "cortex-a72", "-smp", "2", "-m", "8G", "-device", "virtio-net-device,netdev=net0", "-netdev", "user,id=net0,hostfwd=tcp::50022-:22,hostfwd=tcp::8080-:80,hostfwd=tcp::8443-:443", "-drive", "if=virtio,format=qcow2,file=/Users/marcelmrkoch/Development/mk/rcelko.ch-ansible/.vagrant/machines/default/qemu/-pHQzbF6F3I/linked-box.img", "-drive", "if=pflash,format=raw,file=/Users/marcelmrkoch/Development/mk/rcelko.ch-ansible/.vagrant/machines/default/qemu/-pHQzbF6F3I/edk2-aarch64-code.fd,readonly=on", "-drive", "if=pflash,format=raw,file=/Users/marcelmrkoch/Development/mk/rcelko.ch-ansible/.vagrant/machines/default/qemu/-pHQzbF6F3I/edk2-arm-vars.fd", "-chardev", "socket,id=mon0,path=/Users/marcelmrkoch/.vagrant.d/tmp/vagrant-qemu/-pHQzbF6F3I/qemu_socket,server=on,wait=off", "-mon", "chardev=mon0,mode=readline", "-chardev", "socket,id=ser0,path=/Users/marcelmrkoch/.vagrant.d/tmp/vagrant-qemu/-pHQzbF6F3I/qemu_socket_serial,server=on,wait=off", "-serial", "chardev:ser0", "-pidfile", "/Users/marcelmrkoch/Development/mk/rcelko.ch-ansible/.vagrant/machines/default/qemu/-pHQzbF6F3I/qemu.pid", "-parallel", "null", "-monitor", "none", "-display", "none", "-vga", "none", "-daemonize", {:notify=>[:stdout, :stderr, :stdin]}]

Stderr: qemu-system-aarch64: VCPU supports less PA bits (36) than requested by the memory map (40)

[ppggff]

Please try qe.machine = "virt,accel=hvf" ?

[mknet]

Unfortunately, I got the exact same result. Do you want me to push my local state to a repo, so you can try it yourself? Thanks for your help!

[ppggff]

Sorry, I don't have an apple m1 macbook right now. I will try this problem later this weekend.

And for highmem=on, this is the default value, so remove it won't help. (sorry...)

I found qemu 7.0.0 already support -cpu host for apple m1, so no need to use cortex-a72 any more. (from commit 'hvf: arm: Implement -cpu host')

So please try:

config.vm.provider "qemu" do |qe|
    qe.memory = "8G"
    qe.machine = "virt,accel=hvf,highmem=on"
    qe.cpu = "host"
end

[mknet]

Now I got a timeout. :)

==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:50022
    default: SSH username: vagrant
    default: SSH auth method: private key
Timed out while waiting for the machine to boot. This means that
Vagrant was unable to communicate with the guest machine within
the configured ("config.vm.boot_timeout" value) time period.

If you look above, you should be able to see the error(s) that
Vagrant had when attempting to connect to the machine. These errors
are usually good hints as to what may be wrong.

If you're using a custom box, make sure that networking is properly
working and you're able to connect to the machine. It is a common
problem that networking isn't setup properly in these boxes.
Verify that authentication configurations are also setup properly,
as well.

If the box appears to be booting properly, you may want to increase
the timeout ("config.vm.boot_timeout") value.

Could you tell me, how I could start the machine independently? I am a noob when it comes to qemu. Then I can find out, what works and what doesn't. Thx!

[ppggff]

First, I think the 'timeout' error is a good news. It seems the qemu vm is running.

Could you please post your Vagrantfile here? Then I can try to fix it later.

Also, there is an example to debug/fix a timeout problem in the wiki. It may help. You can use nc to connect to the running vm to debug.

---

BTW. To start the qemu vm independently, you could run the qemu-system-aarch64 command yourself. You can get the current running qemu command by command ps -ef|grep qemu after the timeout error. (the vm should already running, then vagrant cannot connect to it by ssh).

The full command may looks like: (figured out by previous errors)

qemu-system-aarch64
  -machine virt,accel=hvf,highmem=on
  -cpu host
  -smp 2
  -m 8G
  -device virtio-net-device,netdev=net0
  -netdev user,id=net0,hostfwd=tcp::50022-:22,hostfwd=tcp::8080-:80,hostfwd=tcp::8443-:443
  -drive if=virtio,format=qcow2,file=/Users/marcelmrkoch/Development/mk/rcelko.ch-ansible/.vagrant/machines/default/qemu/-pHQzbF6F3I/linked-box.img
  -drive if=pflash,format=raw,file=/Users/marcelmrkoch/Development/mk/rcelko.ch-ansible/.vagrant/machines/default/qemu/-pHQzbF6F3I/edk2-aarch64-code.fd,readonly=on
  -drive if=pflash,format=raw,file=/Users/marcelmrkoch/Development/mk/rcelko.ch-ansible/.vagrant/machines/default/qemu/-pHQzbF6F3I/edk2-arm-vars.fd
  -chardev socket,id=mon0,path=/Users/marcelmrkoch/.vagrant.d/tmp/vagrant-qemu/-pHQzbF6F3I/qemu_socket,server=on,wait=off
  -mon chardev=mon0,mode=readline
  -chardev socket,id=ser0,path=/Users/marcelmrkoch/.vagrant.d/tmp/vagrant-qemu/-pHQzbF6F3I/qemu_socket_serial,server=on,wait=off
  -serial chardev:ser0
  -pidfile /Users/marcelmrkoch/Development/mk/rcelko.ch-ansible/.vagrant/machines/default/qemu/-pHQzbF6F3I/qemu.pid
  -parallel null
  -monitor none
  -display none
  -vga none
  -daemonize

You can may a small change to it to start a qemu with terminal to debug. (Make sure there is no running qemu-system-aarch64 process for same vm before start following command)

qemu-system-aarch64 \
  -machine virt,accel=hvf,highmem=on \
  -cpu host \
  -smp 2 \
  -m 8G \
  -device virtio-net-device,netdev=net0 \
  -netdev user,id=net0,hostfwd=tcp::50022-:22,hostfwd=tcp::8080-:80,hostfwd=tcp::8443-:443 \
  -drive if=virtio,format=qcow2,file=/Users/marcelmrkoch/Development/mk/rcelko.ch-ansible/.vagrant/machines/default/qemu/-pHQzbF6F3I/linked-box.img \
  -drive if=pflash,format=raw,file=/Users/marcelmrkoch/Development/mk/rcelko.ch-ansible/.vagrant/machines/default/qemu/-pHQzbF6F3I/edk2-aarch64-code.fd,readonly=on \
  -drive if=pflash,format=raw,file=/Users/marcelmrkoch/Development/mk/rcelko.ch-ansible/.vagrant/machines/default/qemu/-pHQzbF6F3I/edk2-arm-vars.fd \
  -vga none -nographic

[mknet]

The commands do not work for me. I'm gonna try to figure out the corrections myself. But I need some time for it. I will get back to you asap. Thanks for now.

[ppggff]

I tried with apple m1 + macos 12.4 + qemu 7.0.0_1 + the demo centos, it works with following config:

config.vm.provider "qemu" do |qe|
    qe.machine = "virt,accel=hvf,highmem=on"
    qe.cpu = "host"
end

So, the timeout issue is about the box itself.