search

AsahiLinux / muvm

muvm - run programs from your system in a microVM
397 stars 22 forks source link

muvm segfaults running bash #81

Open RossComputerGuy opened 1 month ago

RossComputerGuy commented 1 month ago

Pulled the most recent commit of muvm and the latest release of libkrun and every command segfaults on 6.9.9-asahi

$ RUST_LOG=debug RUST_BACKTRACE=1 ./result/bin/muvm /nix/store/db7vdf04y1xfwmzhs5yac6x1k52g3z70-system-path/bin/sh
[2024-10-10T19:52:05Z DEBUG muvm] sched_setaffinity cpuset=CpuSet { cpu2, cpu3, cpu4, cpu5, cpu6, cpu7, cpu8, cpu9 }
[2024-10-10T19:52:05Z DEBUG muvm] krun_set_vm_config num_vcpus=8 ram_mib=12348
[2024-10-10T19:52:05Z DEBUG muvm::net] passing fd to passt fd=6
[2024-10-10T19:52:05Z DEBUG muvm::env] env vars env={"MESA_LOADER_DRIVER_OVERRIDE": "asahi", "RUST_LOG": "debug", "PATH": "/nix/store/6xhqzildbsvl0v42wp94x3lphj0qdhnb-passt-2024_09_06.6b38f07/bin:/run/wrappers/bin:/home/ross/.local/share/flatpak/exports/bin:/var/lib/flatpak/exports/bin:/home/ross/.nix-profile/bin:/nix/profile/bin:/home/ross/.local/state/nix/profile/bin:/etc/profiles/per-user/ross/bin:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin:/nix/store/cdkid2c9mkc8xcvcq2g9gcsc2yf4r5sd-binutils-wrapper-2.41/bin:/nix/store/gflybx3kqgxffpw65fkkgghgmpm99zlz-pciutils-3.12.0/bin:/nix/store/k8ch2k68in5x9648p5zcdclwb7hpa6n8-pkgconf-wrapper-2.1.1/bin", "HOST_DISPLAY": ":1", "LD_LIBRARY_PATH": "/nix/store/96y8g6hhm0z4rhf4dfpngcwci5d2fvib-pipewire-1.0.7-jack/lib"}
No IPv6 nameserver available for NDP/DHCPv6
[2024-10-10T19:52:05Z DEBUG vmm::linux::vstate] Guest memory starts at 0xfffc55a20000
[2024-10-10T19:52:05Z DEBUG vmm::linux::vstate] Guest memory starts at 0xfffc35a20000
[2024-10-10T19:52:05Z DEBUG vmm::linux::vstate] Guest memory starts at 0xfff870f20000
[2024-10-10T19:52:05Z DEBUG devices::virtio::gpu::device] virtio_gpu: set_shm_region
[2024-10-10T19:52:05Z DEBUG devices::virtio::balloon::event_handler] balloon: activate event
[2024-10-10T19:52:05Z DEBUG devices::virtio::balloon::event_handler] balloon: stats queue event (ignored)
[2024-10-10T19:52:05Z DEBUG devices::virtio::console::event_handler] console: activate event
[2024-10-10T19:52:05Z DEBUG devices::virtio::console::device] Device is ready: initialization ok
[2024-10-10T19:52:05Z DEBUG devices::virtio::rng::event_handler] rng: activate event
[2024-10-10T19:52:05Z DEBUG devices::virtio::rng::event_handler] rng: request queue event
[2024-10-10T19:52:05Z DEBUG devices::virtio::rng::device] rng: process_req()
[2024-10-10T19:52:05Z DEBUG devices::virtio::rng::device] rng: raising IRQ
[2024-10-10T19:52:05Z DEBUG devices::virtio::rng::event_handler] rng: request queue event
[2024-10-10T19:52:05Z DEBUG devices::virtio::rng::device] rng: process_req()
[2024-10-10T19:52:05Z DEBUG devices::virtio::rng::device] rng: raising IRQ
[2024-10-10T19:52:05Z DEBUG devices::virtio::gpu::device] virtio_gpu: GET_shm_region
[2024-10-10T19:52:05Z DEBUG devices::virtio::gpu::device] virtio_gpu: GET_shm_region
[2024-10-10T19:52:05Z DEBUG devices::virtio::gpu::device] virtio_gpu: GET_shm_region
[2024-10-10T19:52:05Z DEBUG devices::virtio::gpu::device] virtio_gpu: GET_shm_region
[2024-10-10T19:52:05Z DEBUG devices::virtio::gpu::event_handler] gpu: activate event
[2024-10-10T19:52:05Z DEBUG devices::virtio::gpu::event_handler] gpu: request queue event
[1]    636089 segmentation fault (core dumped)  RUST_LOG=debug RUST_BACKTRACE=1 ./result/bin/muvm
rowanG077 commented 1 month ago

Can confirm happens to me too on 6.10.6-asahi.

rowanG077 commented 1 month ago

Here is a backtrace if it helps:

$ rust-gdb --args target/debug/muvm /nix/store/7031c1bwpqvmsfnzd0v50vd6v2lb1kni-bash-interactive-5.2p32/bin/sh
...
Reading symbols from target/debug/muvm...
(gdb) handle SIG34 nostop noprint pass noignore
Signal        Stop      Print   Pass to program Description
SIG34         No        No      Yes             Real-time event 34
(gdb) run
Starting program: /home/rowan.goemans/Documents/engineering/muvm/target/debug/muvm /nix/store/7031c1bwpqvmsfnzd0v50vd6v2lb1kni-bash-interactive-5.2p32/bin/sh
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/nix/store/qii6jadf3xrhg4y0f5m09k11lh5ymwnv-glibc-2.40-36/lib/libthread_db.so.1".
[Detaching after vfork from child process 97935]
[New Thread 0xffe394b2ef80 (LWP 97936)]
[New Thread 0xffe39491ef80 (LWP 97937)]
[New Thread 0xffe39470ef80 (LWP 97938)]
[New Thread 0xffe3944fef80 (LWP 97939)]
[New Thread 0xffe3942eef80 (LWP 97940)]
[New Thread 0xffe37fffef80 (LWP 97941)]
[New Thread 0xffe37fdeef80 (LWP 97942)]
[New Thread 0xffe37fbdef80 (LWP 97943)]
[New Thread 0xffe37f9cef80 (LWP 97947)]
[New Thread 0xffe37f7bef80 (LWP 97948)]
[New Thread 0xffe37f5aef80 (LWP 97949)]
[New Thread 0xffe37f39ef80 (LWP 97950)]

Thread 11 "gpu worker" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xffe37f7bef80 (LWP 97948)]
0x0000fffff7a5a710 in free () from /nix/store/qii6jadf3xrhg4y0f5m09k11lh5ymwnv-glibc-2.40-36/lib/libc.so.6
(gdb) bt
#0  0x0000fffff7a5a710 in free () from /nix/store/qii6jadf3xrhg4y0f5m09k11lh5ymwnv-glibc-2.40-36/lib/libc.so.6
#1  0x0000fffff60a470c in dri_destroy () from /nix/store/hjyy897zil6830j3rmj6g14ym1pi6y5i-mesa-24.2.4/lib/libgbm.so.1
#2  0x0000fffff60a3408 in _gbm_device_destroy () from /nix/store/hjyy897zil6830j3rmj6g14ym1pi6y5i-mesa-24.2.4/lib/libgbm.so.1
#3  0x0000fffff62c54c8 in virgl_gbm_fini () from /nix/store/8m9wbqkh0xrw8bhvrkfz18i57pmdlf4s-virglrenderer-1.1.0/lib/libvirglrenderer.so.1
#4  0x0000fffff62c4cbc in vrend_winsys_init () from /nix/store/8m9wbqkh0xrw8bhvrkfz18i57pmdlf4s-virglrenderer-1.1.0/lib/libvirglrenderer.so.1
#5  0x0000fffff62863f0 in virgl_renderer_init () from /nix/store/8m9wbqkh0xrw8bhvrkfz18i57pmdlf4s-virglrenderer-1.1.0/lib/libvirglrenderer.so.1
#6  0x0000fffff7d300c0 in devices::virtio::gpu::worker::Worker::work () from /nix/store/5jx2pwvcplv6dnynkdndhv41mnc9xfpi-libkrun-1.9.5/lib/libkrun.so.1
#7  0x0000fffff7d244e4 in std::sys::backtrace::__rust_begin_short_backtrace () from /nix/store/5jx2pwvcplv6dnynkdndhv41mnc9xfpi-libkrun-1.9.5/lib/libkrun.so.1
#8  0x0000fffff7d24374 in core::ops::function::FnOnce::call_once{{vtable.shim}} () from /nix/store/5jx2pwvcplv6dnynkdndhv41mnc9xfpi-libkrun-1.9.5/lib/libkrun.so.1
#9  0x0000fffff7e4ab30 in std::sys::pal::unix::thread::Thread::new::thread_start () from /nix/store/5jx2pwvcplv6dnynkdndhv41mnc9xfpi-libkrun-1.9.5/lib/libkrun.so.1
#10 0x0000fffff7a486bc in start_thread () from /nix/store/qii6jadf3xrhg4y0f5m09k11lh5ymwnv-glibc-2.40-36/lib/libc.so.6
#11 0x0000fffff7ab4c8c in thread_start () from /nix/store/qii6jadf3xrhg4y0f5m09k11lh5ymwnv-glibc-2.40-36/lib/libc.so.6
afilini commented 1 month ago

Can confirm this happens to me as well, I'm running NixOS on 6.10.6-asahi, although my backtrace is different:

Thread 7 "fc_vcpu 0" received signal SIGSEGV, Segmentation fault.
[Switching to LWP 562010]
0x0000fffff7a67948 in __strlen_generic () from /nix/store/qii6jadf3xrhg4y0f5m09k11lh5ymwnv-glibc-2.40-36/lib/libc.so.6
(gdb) bt
#0  0x0000fffff7a67948 in __strlen_generic () from /nix/store/qii6jadf3xrhg4y0f5m09k11lh5ymwnv-glibc-2.40-36/lib/libc.so.6
#1  0x0000fffff7a61344 in strdup () from /nix/store/qii6jadf3xrhg4y0f5m09k11lh5ymwnv-glibc-2.40-36/lib/libc.so.6
#2  0x0000fff55ad7e138 in dri2_initialize_drm () from /nix/store/23j2wyndh3qr3p5kfhfhgmv8f3y5778x-mesa-24.2.0-drivers/lib/libEGL_mesa.so.0
#3  0x0000fff55ad77f00 in dri2_initialize () from /nix/store/23j2wyndh3qr3p5kfhfhgmv8f3y5778x-mesa-24.2.0-drivers/lib/libEGL_mesa.so.0
#4  0x0000fff55ad64e78 in eglInitialize () from /nix/store/23j2wyndh3qr3p5kfhfhgmv8f3y5778x-mesa-24.2.0-drivers/lib/libEGL_mesa.so.0
#5  0x0000fffff62c5e70 in virgl_egl_init () from /nix/store/8m9wbqkh0xrw8bhvrkfz18i57pmdlf4s-virglrenderer-1.1.0/lib/libvirglrenderer.so.1
#6  0x0000fffff62c4c8c in vrend_winsys_init () from /nix/store/8m9wbqkh0xrw8bhvrkfz18i57pmdlf4s-virglrenderer-1.1.0/lib/libvirglrenderer.so.1
#7  0x0000fffff62863f0 in virgl_renderer_init () from /nix/store/8m9wbqkh0xrw8bhvrkfz18i57pmdlf4s-virglrenderer-1.1.0/lib/libvirglrenderer.so.1
#8  0x0000fffff7d20a28 in devices::virtio::gpu::worker::Worker::work () from /nix/store/7d87rr76ndxfx7939bjg7i1gw5csvrhx-libkrun-1.9.5/lib/libkrun.so.1
#9  0x0000fffff7d5c5f0 in std::sys::backtrace::__rust_begin_short_backtrace () from /nix/store/7d87rr76ndxfx7939bjg7i1gw5csvrhx-libkrun-1.9.5/lib/libkrun.so.1
#10 0x0000fffff7d16448 in core::ops::function::FnOnce::call_once{{vtable.shim}} () from /nix/store/7d87rr76ndxfx7939bjg7i1gw5csvrhx-libkrun-1.9.5/lib/libkrun.so.1
#11 0x0000fffff7e49414 in std::sys::pal::unix::thread::Thread::new::thread_start () from /nix/store/7d87rr76ndxfx7939bjg7i1gw5csvrhx-libkrun-1.9.5/lib/libkrun.so.1
#12 0x0000fffff7a486bc in start_thread () from /nix/store/qii6jadf3xrhg4y0f5m09k11lh5ymwnv-glibc-2.40-36/lib/libc.so.6
#13 0x0000fffff7ab4c8c in thread_start () from /nix/store/qii6jadf3xrhg4y0f5m09k11lh5ymwnv-glibc-2.40-36/lib/libc.so.6
rowanG077 commented 2 days ago

I would really like to get this working on nixos. Is there any information I could provide that would help? Do one of the maintainers know what could be going on? I tried the main branch a few days ago and the same issue persists.

RossComputerGuy commented 2 days ago

At this point, the PR appears to work. It just needs reviewers.

I tried the main branch a few days ago and the same issue persists.

You likely ran into the Mesa issue. You need to use the same exact Mesa as the one on your system.