search

martinpitt / umockdev

Mock hardware devices for creating unit tests and bug reporting
https://launchpad.net/umockdev
GNU Lesser General Public License v2.1
314 stars 58 forks source link

test-umockdev-vala crashes with SIGBUS on sparc64 #218

Closed glaubitz closed 9 months ago

glaubitz commented 1 year ago

On Debian unstable running on sparc64, the test test-umockdev-vala fails with SIGBUS:

==================================== 3/7 =====================================
test:         umockdev:fails-valgrind / umockdev-vala
start time:   20:45:35
duration:     0.86s
result:       killed by signal 10 SIGBUS
command:      LD_LIBRARY_PATH=/home/glaubitz/umockdev/build/:/home/glaubitz/umockdev/build MALLOC_PERTURB_=232 PATH=/home/glaubitz/umockdev/build:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games GI_TYPELIB_PATH=/home/glaubitz/um
ockdev/build G_DEBUG=fatal-warnings,fatal-criticals,gc-friendly TOP_SRCDIR=/home/glaubitz/umockdev MALLOC_CHECK_=3 /home/glaubitz/umockdev/src/umockdev-wrapper /home/glaubitz/umockdev/build/test-umockdev-vala
----------------------------------- stdout -----------------------------------
TAP version 13
# random seed: R02S20c1c80c280731e011fcfe835e71c61f
1..19
# Start of umockdev-testbed-vala tests
# DEBUG: umockdev.vala:123: Created udev test bed /tmp/umockdev.91CLE2
# DEBUG: umockdev.vala:150: Removing test bed /tmp/umockdev.91CLE2
# GLib-DEBUG: unsetenv() is not thread-safe and should not be used after threads are created
ok 1 /umockdev-testbed-vala/empty
# GLib-DEBUG: setenv()/putenv() are not thread-safe and should not be used after threads are created
# DEBUG: umockdev.vala:123: Created udev test bed /tmp/umockdev.7T9KE2
# DEBUG: umockdev.vala:771: umockdev_testbed_uevent: lazily initializing uevent_sender
# DEBUG: umockdev.vala:775: umockdev_testbed_uevent: sending uevent add for device /sys/devices/extkeyboard1
# DEBUG: umockdev.vala:150: Removing test bed /tmp/umockdev.7T9KE2
# GLib-DEBUG: unsetenv() is not thread-safe and should not be used after threads are created
ok 2 /umockdev-testbed-vala/add_devicev
# GLib-DEBUG: setenv()/putenv() are not thread-safe and should not be used after threads are created
# DEBUG: umockdev.vala:123: Created udev test bed /tmp/umockdev.QZFLE2
# DEBUG: umockdev.vala:1297: parsing device description for /devices/myusbhub/cam
# DEBUG: umockdev.vala:1366: creating device /devices/myusbhub/cam (subsystem usb)
# DEBUG: umockdev.vala:1411: create_node_for_device: creating file device /tmp/umockdev.QZFLE2/dev/bus/usb/001/002
# DEBUG: umockdev.vala:771: umockdev_testbed_uevent: lazily initializing uevent_sender
# DEBUG: umockdev.vala:775: umockdev_testbed_uevent: sending uevent add for device /sys/devices/myusbhub/cam
# DEBUG: umockdev.vala:1297: parsing device description for /devices/myusbhub
# DEBUG: umockdev.vala:1366: creating device /devices/myusbhub (subsystem usb)
# DEBUG: umockdev.vala:1411: create_node_for_device: creating file device /tmp/umockdev.QZFLE2/dev/bus/usb/001/001
# DEBUG: umockdev.vala:775: umockdev_testbed_uevent: sending uevent add for device /sys/devices/myusbhub
# DEBUG: umockdev.vala:150: Removing test bed /tmp/umockdev.QZFLE2
# GLib-DEBUG: unsetenv() is not thread-safe and should not be used after threads are created
ok 3 /umockdev-testbed-vala/gudev-query-list
# GLib-DEBUG: setenv()/putenv() are not thread-safe and should not be used after threads are created
# DEBUG: umockdev.vala:123: Created udev test bed /tmp/umockdev.FQ6KE2
# DEBUG: umockdev.vala:771: umockdev_testbed_uevent: lazily initializing uevent_sender
# DEBUG: umockdev.vala:775: umockdev_testbed_uevent: sending uevent add for device /sys/devices/dev1
# DEBUG: umockdev.vala:150: Removing test bed /tmp/umockdev.FQ6KE2
# GLib-DEBUG: unsetenv() is not thread-safe and should not be used after threads are created
ok 4 /umockdev-testbed-vala/fs_ops
# GLib-DEBUG: setenv()/putenv() are not thread-safe and should not be used after threads are created
# DEBUG: umockdev.vala:123: Created udev test bed /tmp/umockdev.1ZXVE2
# DEBUG: umockdev.vala:1297: parsing device description for /devices/mycam
# DEBUG: umockdev.vala:1366: creating device /devices/mycam (subsystem usb)
# DEBUG: umockdev.vala:1411: create_node_for_device: creating file device /tmp/umockdev.1ZXVE2/dev/001
# DEBUG: umockdev.vala:771: umockdev_testbed_uevent: lazily initializing uevent_sender
# DEBUG: umockdev.vala:775: umockdev_testbed_uevent: sending uevent add for device /sys/devices/mycam
# DEBUG: umockdev.vala:150: Removing test bed /tmp/umockdev.1ZXVE2
# GLib-DEBUG: unsetenv() is not thread-safe and should not be used after threads are created
ok 5 /umockdev-testbed-vala/usbfs_ioctl_static
# GLib-DEBUG: setenv()/putenv() are not thread-safe and should not be used after threads are created
# DEBUG: umockdev.vala:123: Created udev test bed /tmp/umockdev.7X2VE2
# DEBUG: umockdev.vala:1297: parsing device description for /devices/mycam
# DEBUG: umockdev.vala:1366: creating device /devices/mycam (subsystem usb)
# DEBUG: umockdev.vala:1411: create_node_for_device: creating file device /tmp/umockdev.7X2VE2/dev/001
# DEBUG: umockdev.vala:771: umockdev_testbed_uevent: lazily initializing uevent_sender
# DEBUG: umockdev.vala:775: umockdev_testbed_uevent: sending uevent add for device /sys/devices/mycam
# GLib-GIO-DEBUG: _g_io_module_get_default: Found default implementation local (GLocalVfs) for ?gio-vfs?
# DEBUG: umockdev.vala:150: Removing test bed /tmp/umockdev.7X2VE2
# GLib-DEBUG: unsetenv() is not thread-safe and should not be used after threads are created
ok 6 /umockdev-testbed-vala/usbfs_ioctl_tree
# GLib-DEBUG: setenv()/putenv() are not thread-safe and should not be used after threads are created
# DEBUG: umockdev.vala:123: Created udev test bed /tmp/umockdev.9HHVE2
# DEBUG: umockdev.vala:1297: parsing device description for /devices/mycam
# DEBUG: umockdev.vala:1366: creating device /devices/mycam (subsystem usb)
# DEBUG: umockdev.vala:1411: create_node_for_device: creating file device /tmp/umockdev.9HHVE2/dev/001
# DEBUG: umockdev.vala:771: umockdev_testbed_uevent: lazily initializing uevent_sender
# DEBUG: umockdev.vala:775: umockdev_testbed_uevent: sending uevent add for device /sys/devices/mycam
# DEBUG: umockdev.vala:150: Removing test bed /tmp/umockdev.9HHVE2
# GLib-DEBUG: unsetenv() is not thread-safe and should not be used after threads are created
ok 7 /umockdev-testbed-vala/usbfs_ioctl_tree_with_default_device
# GLib-DEBUG: setenv()/putenv() are not thread-safe and should not be used after threads are created
# DEBUG: umockdev.vala:123: Created udev test bed /tmp/umockdev.LTMVE2
# DEBUG: umockdev.vala:1297: parsing device description for /devices/mycam
# DEBUG: umockdev.vala:1366: creating device /devices/mycam (subsystem usb)
# DEBUG: umockdev.vala:1411: create_node_for_device: creating file device /tmp/umockdev.LTMVE2/dev/002
# DEBUG: umockdev.vala:771: umockdev_testbed_uevent: lazily initializing uevent_sender
# DEBUG: umockdev.vala:775: umockdev_testbed_uevent: sending uevent add for device /sys/devices/mycam
# DEBUG: umockdev.vala:150: Removing test bed /tmp/umockdev.LTMVE2
# GLib-DEBUG: unsetenv() is not thread-safe and should not be used after threads are created
ok 8 /umockdev-testbed-vala/usbfs_ioctl_tree_override_default_device
# GLib-DEBUG: setenv()/putenv() are not thread-safe and should not be used after threads are created
# DEBUG: umockdev.vala:123: Created udev test bed /tmp/umockdev.VI0UE2
# DEBUG: umockdev.vala:1297: parsing device description for /devices/mycam
# DEBUG: umockdev.vala:1366: creating device /devices/mycam (subsystem usb)
# DEBUG: umockdev.vala:1411: create_node_for_device: creating file device /tmp/umockdev.VI0UE2/dev/001
# DEBUG: umockdev.vala:771: umockdev_testbed_uevent: lazily initializing uevent_sender
# DEBUG: umockdev.vala:775: umockdev_testbed_uevent: sending uevent add for device /sys/devices/mycam
# DEBUG: umockdev.vala:150: Removing test bed /tmp/umockdev.VI0UE2
# GLib-DEBUG: unsetenv() is not thread-safe and should not be used after threads are created
ok 9 /umockdev-testbed-vala/usbfs_ioctl_tree_xz
# slow test /umockdev-testbed-vala/usbfs_ioctl_tree_xz executed in 0.60 secs
# GLib-DEBUG: setenv()/putenv() are not thread-safe and should not be used after threads are created
# DEBUG: umockdev.vala:123: Created udev test bed /tmp/umockdev.AATGE2
# DEBUG: umockdev.vala:1297: parsing device description for /devices/pci0000:00/0000:00:14.0/usb1/1-3
# DEBUG: umockdev.vala:1366: creating device /devices/pci0000:00/0000:00:14.0/usb1/1-3 (subsystem usb)
# DEBUG: umockdev.vala:1411: create_node_for_device: creating file device /tmp/umockdev.AATGE2/dev/bus/usb/001/011
# DEBUG: umockdev.vala:771: umockdev_testbed_uevent: lazily initializing uevent_sender
# DEBUG: umockdev.vala:775: umockdev_testbed_uevent: sending uevent add for device /sys/devices/pci0000:00/0000:00:14.0/usb1/1-3
# DEBUG: umockdev.vala:1297: parsing device description for /devices/pci0000:00/0000:00:14.0/usb1
# DEBUG: umockdev.vala:1366: creating device /devices/pci0000:00/0000:00:14.0/usb1 (subsystem usb)
# DEBUG: umockdev.vala:1411: create_node_for_device: creating file device /tmp/umockdev.AATGE2/dev/bus/usb/001/001
# DEBUG: umockdev.vala:775: umockdev_testbed_uevent: sending uevent add for device /sys/devices/pci0000:00/0000:00:14.0/usb1
# DEBUG: umockdev.vala:1297: parsing device description for /devices/pci0000:00/0000:00:14.0
# DEBUG: umockdev.vala:1366: creating device /devices/pci0000:00/0000:00:14.0 (subsystem pci)
# DEBUG: umockdev.vala:775: umockdev_testbed_uevent: sending uevent add for device /sys/devices/pci0000:00/0000:00:14.0
==============================================================================

This can easily be reproduced on Debian's new sparc64 porterbox stadler and is also visible in the last build log [1].

Unfortunately, I have no experience with Vala, so I have no idea how to debug this.

[1] https://buildd.debian.org/status/fetch.php?pkg=umockdev&arch=sparc64&ver=0.17.18-1&stamp=1699723932&raw=0

martinpitt commented 9 months ago

Hmm, I was going to take a look, but unfortunately stadler.debian.net doesn't respond right now :cry:

glaubitz commented 9 months ago

Hmm, I was going to take a look, but unfortunately stadler.debian.net doesn't respond right now 😢

It's up and running, but the IPv6 connection does not work properly. Try ssh -4 stadler.debian.net to use IPv4-only.

martinpitt commented 9 months ago

Reproducer with running single test after building:

meson setup c
meson test -C c
gdb --args env LD_LIBRARY_PATH=c LD_PRELOAD=libumockdev-preload.so.0 c/test-umockdev-vala -p /umockdev-testbed-vala/usbfs_ioctl_pcap

Unfortunately the stack trace is absolutely useless:

#0  0xfff80001020259a4 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

and so is strace.

This is the only test which fails. If I disable it then everything else succeeds.

I'm actually surprised that this test passes on a lot of architectures -- the pcap file was recorded on x86_64, and there may be some unaligned access there. It's equally possible that some of the buffers allocated in the test are not aligned enough? (but I don't think vala gives you control over this).

Stepping through with gdb it fails in line 580:

assert_cmpint (Posix.ioctl (fd, Ioctl.USBDEVFS_REAPURB, ref urb_reap), CompareOperator.EQ, 0);

The struct usbdevfs_urb is fairly complicated, but /usr/include/linux/usbdevice_fs.h has a matching C declaration, except for void *buffer → uint8[] buffer. I don't know enough about platform specific alignment etc. to know if that is relevant.

So I think the more pressing issue is the FTBFS, so I'll just skip the test on sparc64 for the time being. Actually fixing that requires a compiler expert, and it may even be that pcap files are not platform independent.

martinpitt commented 9 months ago

Bad timing -- I just released 0.18.0 this morning, but the next release will come at some point :grin:

benzea commented 9 months ago

Hmm, I think in principle the test can be made to work both in big/little endian. But, yes, I think you are right and the pcap code is broken on both big endian and likely also on machines that cannot handle unaligned accesses (which explains the SIGBUS after all).

More specifically, I think the problem are the accesses into usb_header_mmapped (urb_hdr in umockdev-pcap.vala). This data is mmapped and also little endian, but the code just directly reads all those entries.

Note that this code is running in a separate thread. So if you wanted to debug stepping through the test will not narrow it down on that side. I think you need to break on IoctlUsbPcapHandler::handle_ioctl and step through the code there.

EDIT: Hm, thinking about it, I suspect the mmap'ed data should be correctly aligned. In which case, we probably only fall over due to big/little endian handling.