robertkirkman
commented
2 weeks ago Open robertkirkman opened 2 weeks ago
robertkirkman
commented
2 weeks ago robertkirkman
commented
2 weeks ago I made a further observation, while watching htop within QEMU while trying to boot redroid:
/system/bin/servicemanager is using a lot of emulated CPU, but it continues to even after a very long time and no further kernel messages print, for much longer than any task in Arch Linux ARM takes, even intensive ones
robertkirkman
commented
2 weeks ago And finally, the reason why userspace emulation isn't sufficient is because it doesn't work on certain specific apps. One example is Termux. The aarch64 build of Termux does not work within amd64 redroid that has a userspace ARM emulator, so, the ultimate goal here is run the aarch64 build of Termux on amd64 hardware
robertkirkman
commented
2 weeks ago Actually, it really was just loading! This time, I waited for 7 hours and it finally progressed past that part and booted fully, now I am able to connect with scrcpy and use redroid inside the full system emulator! After fully booting, the CPU performance in redroid is indeed about 50% the speed of a Galaxy S III. That is sufficient for my needs.
However it would be really nice to be able to have similar GPU performance to everything else, by getting surfaceflinger to utilize the same virtio GPU that the Arch Linux ARM Xorg is able to. But, unfortunately that might be very difficult, because I see this in the output from qemu-system-aarch64 if I try to use androidboot.redroid_gpu_mode=host, and the scrcpy viewport shows a black screen with some artifacts (software rendering mode androidboot.redroid_gpu_mode=guest works though)
Also, if Xorg is running in the KMS+DRM viewport (the QEMU SDL window) when I try to launch redroid with host gpu mode, Xorg freezes. I guess this might indicate it is a bug with QEMU or the virtio driver itself. virtio-gpu-gl-pci does not require CPU virtualization to be enabled. It works with Xorg even with QEMU in CPU emulation mode.
Here is a copy of everything from the .tgz file when redroid booted successfully but has a black screen in gpu host mode (huge paste) https://defuse.ca/b/cjECnfsiwzRQJoP4ebeX8g
After the 7 hours have passed, kernel log messages begin printing quickly again and redroid becomes possible to connect to using scrcpy soon after. It looks to me like there is one spot in the boot process that takes much longer than everything else to run inside QEMU, but it works ok when finished.
In gpu guest mode, it is now working pretty good after I wait the time for it to boot, so I will use that for now.
robertkirkman
commented
2 weeks ago To summarize:
zhouziyang
commented
2 weeks ago Well, the full system emulation is strongly not recommended (painfully slow w/ compatibility issues). How about native bridge (redroid already bundle this since redroid11)? Try a arm64 native platform if native bridge not suffice.
robertkirkman
commented
2 weeks ago I will post a screenshot of the result I achieved that is the reason why I use this. When I tried the native bridge redroid, it was unable to reproduce this screenshot. The full system emulator causes this to work though. (Termux is one of the actual apps I need, and I need it to be the aarch64 version of Termux specifically. That's the reason why I need this method)
zhouziyang
commented
1 week ago Why not just use Termux for x86_64?
BTW, I believe the full system emulation is painfully slow, but should not that slow (you wait about 7 hours to boot redroid?). Again, strongly recommend native arm64 platform if native bridge not suffice.
Describe the bug I have tried a lot of things to try to get the aarch64 redroid to run within
qemu-system-aarch64that is running on an amd64 hardware host, and trying to customize the distro runningdockerwithin QEMU until it is able to run redroid, but I haven't ever been successful. EDIT: I was successful today after trying again, so now it only has some problems like host GPU mode not workingIf this isn't a supported use-case that is ok. I just feel that someday, it will work, because my specific
qemu-system-aarch64instance has stronger performance than average due to the host having strong hardware, Ryzen 9 5950X and Radeon RX 5700 XT, which are utilized by the-smp 32and-device virtio-gpu-gl-pciarguments toqemu-system-aarch64:QEMU invocation
```bash qemu-system-aarch64 -M virt \ -m 16384 \ -cpu cortex-a72 \ -smp 32 \ -drive if=pflash,media=disk,format=raw,cache=writethrough,file=flash0.img \ -drive if=pflash,media=disk,format=raw,cache=writethrough,file=flash1.img \ -drive if=none,file=archarmvm.qcow2,format=qcow2,id=hd0 \ -device virtio-scsi-pci,id=scsi0 \ -device scsi-hd,bus=scsi0.0,drive=hd0,bootindex=1 \ -nic user,model=virtio-net-pci,hostfwd=tcp::2222-:22 \ -device virtio-rng-device \ -device virtio-balloon-device \ -device virtio-serial-device \ -device virtio-tablet-device \ -object cryptodev-backend-builtin,id=cryptodev0 \ -device virtio-crypto-pci,id=crypto0,cryptodev=cryptodev0 \ -device virtio-gpu-gl-pci,xres=1920,yres=1080 \ -display sdl,gl=on,show-cursor=on \ -device qemu-xhci,id=xhci \ -device usb-host,bus=xhci.0,hostdevice=/dev/bus/usb/003/008 \ -device usb-host,bus=xhci.0,hostdevice=/dev/bus/usb/003/009 ```The way the
dockerservice runner, Arch Linux ARM, is installed withinqemu-system-aarch64is documented here. It is capable of running many aarch64 GNU/Linux apps with usable framerate, and is also capable of running other docker containers besides redroid. That is why I hope that performance might be sufficient to run redroid.I enabled the necessary kernel flags within the QEMU guest like this:
Arch Linux ARM custom kernel steps
```bash sudo pacman -S base-devel git git clone https://github.com/archlinuxarm/PKGBUILDs.git cd PKGBUILDs/core/linux-aarch64/ git apply -v << 'EOF' --- a/core/linux-aarch64/PKGBUILD +++ b/core/linux-aarch64/PKGBUILD @@ -28,7 +28,7 @@ md5sums=('b9828ed78dae306e3d90643cd5cdb8f1' 'f65db321ef86dcccf3d27d26023c6438' '7b08a199a97e3e2288e5c03d8e8ded2d' 'c9d4e392555b77034e24e9f87c5ff0b3' - '22283f60f34dddb1d4b6748482b6c86c' + 'SKIP' '7c97cf141750ad810235b1ad06eb9f75' '61c5ff73c136ed07a7aadbf58db3d96a' '584777ae88bce2c5659960151b64c7d8' --- a/core/linux-aarch64/config +++ b/core/linux-aarch64/config @@ -8701,11 +8701,13 @@ CONFIG_DMA_ENGINE_RAID=y # CONFIG_SYNC_FILE=y # CONFIG_SW_SYNC is not set -# CONFIG_UDMABUF is not set +CONFIG_UDMABUF=y # CONFIG_DMABUF_MOVE_NOTIFY is not set # CONFIG_DMABUF_DEBUG is not set # CONFIG_DMABUF_SELFTESTS is not set -# CONFIG_DMABUF_HEAPS is not set +CONFIG_DMABUF_HEAPS=y +CONFIG_DMABUF_HEAPS_SYSTEM=y +CONFIG_DMABUF_HEAPS_CMA=y # CONFIG_DMABUF_SYSFS_STATS is not set # end of DMABUF options @@ -10565,7 +10567,10 @@ CONFIG_RAS=y # # Android # -# CONFIG_ANDROID_BINDER_IPC is not set +CONFIG_ANDROID_BINDER_IPC=y +CONFIG_ANDROID_BINDERFS=y +CONFIG_ANDROID_BINDER_DEVICES="binder,hwbinder,vndbinder" +CONFIG_ANDROID_BINDER_IPC_SELFTEST=y # end of Android CONFIG_LIBNVDIMM=y EOF # Emulator can compile the kernel for itself quickly and not too slow! echo 'MAKEFLAGS="-j32"' | sudo tee -a /etc/makepkg.conf makepkg -s sudo pacman -U linux-aarch64-6.9.6-1-aarch64.pkg.tar.xz linux-aarch64-headers-6.9.6-1-aarch64.pkg.tar.xz sudo reboot ```Then I am trying to run redroid like this:
Then
adb connect localhostandadb devices -lshowsoffline.make sure the required kernel modules present
grep binder /proc/filesystemsgrep ashmem /proc/misc(optional)Collect debug logs
curl -fsSL https://raw.githubusercontent.com/remote-android/redroid-doc/master/debug.sh | sudo bash -s -- redroidSee comment, and kernel config omitted because it is too big. My kernel config is this but with the patch described above applied.
Screenshots If applicable, add screenshots to help explain your problem.
Single-thread performance gets 0.26 in this benchmark, which is comparable to the score of an older smartphone (Samsung Galaxy S III Sprint SPH-L710) - usable, and not frozen.
For this reason, I hope my QEMU has sufficient performance to run redroid, and the problem must be caused by a small detail somewhere, which can hopefully be solved.