Startup error after modifying configuration.

[nongze]

When I added configuration to Docker Compose.yml to modify RAM and CPU, there was a startup error, whether it was the first creation or modification after creation.

    environment:
      VERSION: "win11"
      RAM_SIZE: "4G"
      CPU_CORES: "4"

root@orangepi5:/root/win# docker-compose up
[+] Building 0.0s (0/0)                                                                                                      docker:default
[+] Running 1/1
 ✔ Container windows  Recreated                                                                                                        0.2s 
Attaching to windows
windows  | ❯ Starting Windows for Docker v1.08...
windows  | ❯ For support visit https://github.com/dockur/windows-arm
windows  | 
windows  | ❯ Booting Windows using QEMU emulator version 8.2.1 ...
windows  | ❯ ERROR: qemu-system-aarch64: Failed to put registers after init: Invalid argument
windows exited with code 0

If the default configuration is used, there will be no errors.

[kroese]

I cannot think of a reason why this happens. Because even without setting these values in your Dockerfile, they are still set to the defaults (4G and 2 cores) so if anything was wrong with the order I set them for example, it would also happen with the defaults.

I will see if I can reproduce this and fix it, but currently I have no ARM hardware available anymore (used my Pi5 for something else now), so it might take some time before I can fix this.

[yuunnn-w]

I cannot think of a reason why this happens. Because even without setting these values in your Dockerfile, they are still set to the defaults (4G and 2 cores) so if anything was wrong with the order I set them for example, it would also happen with the defaults.

I will see if I can reproduce this and fix it, but currently I have no ARM hardware available anymore (used my Pi5 for something else now), so it might take some time before I can fix this.

I suspect the issue might be due to the qemu version you are using being too old, leading to abnormal parameter passing. The error message is: ERROR: qemu-system-aarch64: Failed to put registers after init: Invalid argument . Moreover, the error occurs regardless of whether the parameters in the configuration file are removed or not. The qemu version we are using is 8.2.1, and the hardware we use is OrangePi 5 Plus.

[samirdjelal]

I've got the same issue on OrangePi 5 Plus, I've used the default docker-compose without any changes or custom env vars

windows  | ❯ Extracting Windows 11 for ARM bootdisk...
windows  | ❯ Extracting Windows 11 for ARM environment...
windows  | ❯ Extracting Windows 11 for ARM setup...
windows  | ❯ Extracting Windows 11 for ARM image...
windows  | ❯ Adding XML file for automatic installation...
windows  | ❯ Building Windows 11 for ARM image...
windows  | ❯ Creating a 64G growable disk image in raw format...
windows  | ❯ Booting Windows using QEMU emulator version 8.2.1 ...
windows  | ❯ ERROR: qemu-system-aarch64: Failed to put registers after init: Invalid argument
windows  | ❯ Starting Windows for Docker v1.08...
windows  | ❯ For support visit https://github.com/dockur/windows-arm

[kroese]

For anyone experiencing this issue, can you add:

environment:
  DEBUG: "Y"

to your compose file. And then copy and paste the QEMU commandline arguments here. And also a copy of the arguments when it does not give that error. So we can compare both and see what the difference is between them?

[Explosion-Scratch]

@kroese Happening to me as well:

windows  | ❯ Starting Windows for Docker v1.09...
windows  | ❯ For support visit https://github.com/dockur/windows-arm
windows  |
windows  | ❯ Host: c84093f0d383  IP: 172.24.0.2  Gateway: 172.24.0.1  Interface: eth0  MAC: 02:7E:CC:0F:F0:80
windows  | nameserver 127.0.0.11
windows  |
windows  | ++ /usr/sbin/dnsmasq --dhcp-range=20.20.20.21,20.20.20.21 --dhcp-host=02:7E:CC:0F:F0:80,,20.20.20.21,QEMU,infinite --dhcp-option=option:netmask,255.255.255.0 --dhcp-option=option:dns-server,20.20.20.1 --dhcp-option=option:router,20.20.20.1 --address=/host.lan/20.20.20.1
windows  |
windows  | ❯ Booting Windows using QEMU emulator version 8.2.1 ...
windows  | Arguments: -nodefaults -cpu host -smp 4,sockets=1,dies=1,cores=4,threads=1 -m 4G -machine type=virt,secure=off,dump-guest-core=off,accel=kvm -enable-kvm -display vnc=:0,websocket=5700 -device ramfb -monitor telnet:localhost:7100,server,nowait,nodelay -daemonize -D /run/shm/qemu.log -pidfile /run/shm/qemu.pid -name windows,process=windows,debug-threads=on -serial pty -device qemu-xhci,id=xhci -device usb-kbd -device usb-tablet -netdev tap,ifname=qemu,script=no,downscript=no,id=hostnet0 -device virtio-net-pci,romfile=,netdev=hostnet0,mac=02:7E:CC:0F:F0:80,id=net0 -object iothread,id=io2 -drive id=cdrom0,media=cdrom,if=none,format=raw,readonly=on,file=/storage/win11arm64.iso -device virtio-scsi-pci,id=scsi0,iothread=io2,addr=0x5 -device scsi-cd,bus=scsi0.0,drive=cdrom0,bootindex=10 -drive id=cdrom1,media=cdrom,if=none,format=raw,readonly=on,file=/run/drivers.iso -device usb-storage,drive=cdrom1 -drive file=/storage/data.img,if=none,id=drive-userdata,format=raw,cache=none,aio=native,discard=on,detect-zeroes=on -device virtio-scsi-pci,id=hw-userdata,iothread=io2,bus=pcie.0,addr=0xa -device scsi-hd,bus=hw-userdata.0,channel=0,scsi-id=0,lun=0,drive=drive-userdata,id=userdata,rotation_rate=1,bootindex=3 -drive file=/storage/windows.rom,if=pflash,unit=0,format=raw,readonly=on -drive file=/storage/windows.vars,if=pflash,unit=1,format=raw -device virtio-balloon-pci,id=balloon0,bus=pcie.0,addr=0x4 -object rng-random,id=objrng0,filename=/dev/urandom -device virtio-rng-pci,rng=objrng0,id=rng0,bus=pcie.0,addr=0x1c
windows  |
windows  | ❯ ERROR: qemu-system-aarch64: Failed to put registers after init: Invalid argument
windows exited with code 0

`docker-compose.yml`

```yml version: "3" services: windows: container_name: windows image: dockurr/windows-arm environment: VERSION: "win11" RAM_SIZE: "4G" CPU_CORES: "4" DEBUG: "Y" volumes: - /home/tjs/Media/Windows:/storage devices: - /dev/kvm cap_add: - NET_ADMIN ports: - 8006:8006 - 3389:3389/tcp - 3389:3389/udp stop_grace_period: 2m restart: on-failure ```

[Explosion-Scratch]

I also removed all environment variables except for VERSION and DEBUG and it's still happening.

[Explosion-Scratch]

I'm also running the Orange Pi 5B using Armbian

Screenfetch

``` ❯ screenfetch ./+o+- tjs@tjsorange yyyyy- -yyyyyy+ OS: Ubuntu 22.04 jammy ://+//////-yyyyyyo Kernel: aarch64 Linux 5.10.160-legacy-rk35xx .++ .:/++++++/-.+sss/` Uptime: 3d 1h 28m .:++o: /++++++++/:--:/- Packages: 1802 o:+o+:++.`..```.-/oo+++++/ Shell: zsh 5.8.1 .:+o:+o/. `+sssoo+/ Disk: 1.5T / 6.0T (26%) .++/+:+oo+o:` /sssooo. CPU: ARM Cortex-A55 Cortex-A76 @ 8x 1.8GHz /+++//+:`oo+o /::--:. RAM: 4996MiB / 15718MiB \+/+o+++`o++o ++////. .++.o+++oo+:` /dddhhh. .+.o+oo:. `oddhhhh+ \+.++o+o``-````.:ohdhhhhh+ `:o+++ `ohhhhhhhhyo++os: .o:`.syhhhhhhh/.oo++o` /osyyyyyyo++ooo+++/ ````` +oo+++o\: `oo++. ```

[kroese]

I can find very little information about this error. I suspect this is some incompatibility between the OrangePI kernel and KVM, because all of you are running into this problem on a OrangePI device.

From this post: https://forum.armbian.com/topic/29785-orange-pi-5-plus-fails-to-start-ovmf-efi-firmware-under-kvm/ it seems like it can be solved by upgrading the OrangePI to the newer Debian Trixie distribution (which has a newer kernel I guess).

[stlorenz]

I have the same on a Rockchip Device Turing RK1

This has the same chip as the OrangePi RK3588, will test the upgrade.

[stlorenz]

The update runs into errors: Processing triggers for man-db (2.10.2-1) ... perl: /lib/aarch64-linux-gnu/libc.so.6: versionGLIBC_2.36' not found (required by /lib/aarch64-linux-gnu/libcrypt.so.1)` Same for the bash package

[morsDeiGH]

Rockchip RK3588 have 4 core Cortex-A76 and 4 core Cortex-A55. If disable 4 core Cortex-A76 or 4 core Cortex-A55 and after this container running normally, but with both core, we get an error "qemu-system-aarch64: Failed to put registers after init: Invalid argument".

1. sudo chcpu --disable 0,1,2,3
2. docker compose up
3. sudo chcpu --enable 0,1,2,3

The problem seems to be due to different kernels

[rogerdk]

I have come across the same problem. I'm also using an OrangePi5 Could I pass custom parameters for testing? in this thread (https://github.com/Joshua-Riek/ubuntu-rockchip/issues/731#issuecomment-2127360776) they propose to use something like: --vcpus=4,cpuset=4-7.

However, I have tried to disable the cpus before booting and I cannot avoid this error:

'ERROR: Timeout while waiting for QEMU to boot the machine!'

even though the cpu seems high. it's as if despite not being able to connect the windows runs.

[kroese]

@rogerdk That thread is about libvirt which is a different software, the only similarity is that it also uses QEMU under the hood. But you cannot take parameters for that software, and use them with QEMU or this container, because we dont use libvirt at all.

The problem is that the OrangePi has a mix of different cores (big and little) and that KVM fails to handle it.

Maybe if you set it like this:

    environment:
      CPU_CORES: "1"

it will work, because its limited to a single core so there will be no mixing of types, but I am not sure.

The real solution will probably be to do some CPU pinning via QEMU numa parameters like this:

    environment:
      ARGUMENTS: "-numa node,cpus=0-2"

But I am not sure exactly which command will be needed, its explained on https://www.qemu.org/docs/master/system/qemu-manpage.html but its is very complicated.

So the solution by @morsDeiGH looks much easier, to disable the little cores outside of QEMU/the container, so that you dont need to do any CPU pinning anymore.

[sund3RRR]

I think the easiest solution is to override /run/entry.sh and change this line:

- { qemu-system-aarch64 ${ARGS:+ $ARGS} >"$QEMU_OUT" 2>"$QEMU_LOG"; rc=$?; } || :
+ { taskset -c $CPU_TASK_SET qemu-system-aarch64 ${ARGS:+ $ARGS} >"$QEMU_OUT" 2>"$QEMU_LOG"; rc=$?; } || :

Then add this env to the environment section of compose file:

environment:
  - CPU_TASK_SET: "4,5,6,7"

[kroese]

@sund3RRR I will create a new release, with this feature added, thanks!

[sund3RRR]

Off-topic. Great work, man, thank you for windows-arm!

I don't quite understand qemu-kvm related stuff, but is it real to passthrough mali-g610 gpu? Or at least accelerate 3d with virgl adapter? I want to speed up rdp session because cpu performance is not enough :(

[kroese]

@sund3RRR I want to detect the Orange Pi, so the flag will be automaticly applied. Do you know an easy way? For example, I can check if the kernel name ends with -rk35xx but I dont know if all Orange Pi's always use a kernel with that name?

[sund3RRR]

@sund3RRR I want to detect the Orange Pi, so the flag will be automaticly applied. Do you know an easy way? For example, I can check if the kernel name ends with -rk35xx but I dont know if all Orange Pi's always use a kernel with that name?

Hmmm. I don't know. I think this check is not enough. At least my orangepi's kernel name is 6.1.0-1014-rockchip

And technically orangepi can run mainline linux since 6.8 kernel version with edk2-rk3588 uefi

[kroese]

Can you show me how it detects the CPU in the Docker log (the first line where it says CPU/RAM/etc)

[sund3RRR]

Can you show me how it detects the CPU in the Docker log (the first line where it says CPU/RAM/etc)

Sure

❯ Starting Windows for Docker v2.09...
❯ For support visit https://github.com/dockur/windows-arm
❯ CPU: Cortex A55 | RAM: 6/8 GB | DISK: 362 GB (ext4) | HOST: 6.1.0-1014-rockchip...

[kroese]

Thanks! I created a new release now (v2.10), which should have the issue fixed automaticly when it detects the RK3588 chipset.

Can you please run it and see if it works now (without adding that environment variable).

[sund3RRR]

It works! Well done :)

[sund3RRR]

P.S. It doesn't work with base qemux/qemu-arm image due to

❯ Booting image using QEMU v8.2.4...
taskset: failed to execute exec: No such file or directory

Need to patch

- taskset -c "$CPU_PIN" exec qemu-system-aarch64 ${ARGS:+ $ARGS}
+ taskset -c "$CPU_PIN" qemu-system-aarch64 ${ARGS:+ $ARGS}

[nongze]

Can we support 8 cores in this way？

environment:
  - CPU_TASK_SET: "0,1,2,34,5,6,7"

[kroese]

@sund3RRR Thanks! Should be fixed now in qemu-arm v2.05. Let me know if it works okay now.

@nongze No, the whole problem is that it only works with 0 till 4 OR 4 till 7 (because they have different speeds).

[nongze]

After my attempt, it is feasible.

root@orangepi5:~# cat win/docker-compose.yml
version: "3"
services:
  windows:
    environment:
      CPU_TASK_SET: "1,2,3,4,5,6,7"
      VERSION: "win11"
      RAM_SIZE: "4G"
      CPU_CORES: "8"
    device_cgroup_rules:
      - 'c *:* rwm'
    container_name: windows
    image: dockurr/windows-arm
    devices:
      - /dev/kvm
    cap_add:
      - NET_ADMIN
    ports:
      - 8006:8006
      - 3389:3389/tcp
      - 3389:3389/udp
    stop_grace_period: 2m
    restart: on-failure
    networks:
      macnet0:
        ipv4_address: 192.168.1.100

networks:
  macnet0:
    external: true

[sund3RRR]

After my attempt, it is feasible.

root@orangepi5:~# cat win/docker-compose.yml
version: "3"
services:
  windows:
    environment:
      CPU_TASK_SET: "1,2,3,4,5,6,7"
      VERSION: "win11"
      RAM_SIZE: "4G"
      CPU_CORES: "8"
    device_cgroup_rules:
      - 'c *:* rwm'
    container_name: windows
    image: dockurr/windows-arm
    devices:
      - /dev/kvm
    cap_add:
      - NET_ADMIN
    ports:
      - 8006:8006
      - 3389:3389/tcp
      - 3389:3389/udp
    stop_grace_period: 2m
    restart: on-failure
    networks:
      macnet0:
        ipv4_address: 192.168.1.100

networks:
  macnet0:
    external: true

What image and kernel are you using on host?

[kroese]

@nongze That doesnt work. You set CPU_CORES to 8, that means it will create 8 "virtual" cores from the 4 real cores. You can even create a 100 core machine that way, but you should never set CPU_CORES higher than the amount of real cores, because it will make the system slower instead of faster.

Also you set CPU_TASK_SET but I dont use that name nowhere in my code, so it will be totally ignored that value. I use the name CPU_PIN.

So if you want to do a real test, rename CPU_TASK_SET to CPU_PIN and see if it still works.

[nongze]

I have given up, four cores are enough. Off topic: Can I replace the image with another Linux system or have other projects?

[sund3RRR]

I have given up, four cores are enough. Off topic: Can I replace the image with another Linux system or have other projects?

You can use qemux/qemu-arm

[kroese]

@nongze This Windows container is a spin-off of https://github.com/qemus/qemu-arm , so to run Linux-like VM's you can use that one.

[nongze]

I have another question, how do you optimize Windows 11? I run it on a regular QEMU-KVM and it always restarts. Is there a download link for the system image？

[nongze]

I want to use it in a PVE virtual environment.