On RPi 4, current udev makes Matron unable to discover Grids

ambv commented 1 year ago

Environment

Hardware: a CS4270 Shield XL with a Raspberry Pi 4B. Image version: norns220306 base updated to 230614. Kernel: unchanged 5.10.92-18-v7l-g458e2253667a from the 220306 image

Problem

When you run apt update and upgrade udev (along with libudev and libudev-dev) to 247.3-7+rpi1+deb11u2 from the version that ships with 220306 (247.3-6+rpi1), Matron is no longer able to recognize any 2021+ Grid.

Before:

Sep 27 10:20:51 norns ws-wrapper[495]: tty appears to be grid-st
Sep 27 10:20:51 norns ws-wrapper[495]: monome device appears to be a grid; rows=16; cols=16; quads=4
Sep 27 10:20:51 norns ws-wrapper[495]: grid added:        3        monome zero m5032760        m5032760

After:

Sep 27 10:34:32 shield ws-wrapper[474]: device monitor: unmatched tty device

The kernel discovers the device just fine after the update:

Sep 27 10:34:31 shield kernel: usb 1-1.1: new full-speed USB device number 3 using xhci_hcd
Sep 27 10:34:32 shield kernel: usb 1-1.1: New USB device found, idVendor=cafe, idProduct=4001, bcdDevice= 1.00
Sep 27 10:34:32 shield kernel: usb 1-1.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
Sep 27 10:34:32 shield kernel: usb 1-1.1: Product: grid
Sep 27 10:34:32 shield kernel: usb 1-1.1: Manufacturer: monome
Sep 27 10:34:32 shield kernel: usb 1-1.1: SerialNumber: m5032760
Sep 27 10:34:32 shield mtp-probe[1007]: checking bus 1, device 3: "/sys/devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb1/1-1/1-1.1"
Sep 27 10:34:32 shield mtp-probe[1007]: bus: 1, device: 3 was not an MTP device

Diagnosis

It turns out the device is being recognized as before the update, but udev now reports a different child in the device tree from udev_enumerate_*. In turn, when matron/device/device_monitor.c receives a udev_device*, that node doesn't contain the ID_VENDOR and ID_MODEL properties that is_dev_monome_grid() expects. If that function went up the tree, just like get_device_name() does, it would find the correct device and identify the Grid just fine.

This issue happens both when connecting Grids directly to the Raspberry Pi, and when connecting via a USB hub. It happens on both the 2021 Grid 128 as well as on Grid Zero.

Workaround

If you need other packages upgraded, pin the working udev first by running:

$ sudo apt-mark hold udev libudev1 libudev-dev

This will allow other packages to upgrade while holding back those. This functionality is already used to keep the kernel from upgrading automatically.

At least as of tonight, this allows an otherwise full apt upgrade to keep working.

The issue is not present on stock Norns

This doesn't happen on the same image, kernel, and udev version on stock Norns. In this case everything keeps working as before.

tehn commented 1 year ago

thanks for finding this, it could help future seekers.

generally we don't advise people tweak base OS, which includes running apt-get, basically because we haven't tested what might come down the pipe. a large motivation for norns' existence is a stable OS, whereupon we can control the rate of updates, so we don't need to worry about people updating their systems and everything breaking (we have over a decade's experience with this on MacOS trying to support grids.)

catfact commented 1 year ago

yeah i'd wonder if there is some other motivation for updating udev. we could pin it, but then we'd want to pin a lot of other packages too :)

monome / norns-image