search

raspberrypi / linux

Kernel source tree for Raspberry Pi-provided kernel builds. Issues unrelated to the linux kernel should be posted on the community forum at https://forums.raspberrypi.com/
Other
11.05k stars 4.96k forks source link

Kernel 5.15 / 5.14 Raspberry Pi 3: No USB #4707

Closed mkreisl closed 2 years ago

mkreisl commented 2 years ago

I noticed this with kernel 5.14 and now again with 5.15: There is no USB present. And therefore of course no Ethernet and a USB keyboard does not work either.

WLAN works fine, so at least I can access it. To me it looks like the whole USB controller and everything connected to it is switched off (no power). So I looked at /sys/devices/platform/soc/3f980000.usb/ and the directory tree looks completely different than on a working system with 5.4 kernel:

Kernel 5.15:

root@kmxbilr2 /usr/local/sbin # ll /sys/devices/platform/soc/3f980000.usb/
insgesamt 0
-rw-r--r-- 1 root root 4096 17. Nov 19:24 driver_override
-r--r--r-- 1 root root 4096 17. Nov 19:24 modalias
lrwxrwxrwx 1 root root    0 17. Nov 19:24 of_node -> ../../../../firmware/devicetree/base/soc/usb@7e980000
drwxr-xr-x 2 root root    0 17. Nov 19:18 power
lrwxrwxrwx 1 root root    0  1. Jan 1970  subsystem -> ../../../../bus/platform
lrwxrwxrwx 1 root root    0 17. Nov 19:24 supplier:platform:phy -> ../../../virtual/devlink/platform:phy--platform:3f980000.usb
lrwxrwxrwx 1 root root    0 17. Nov 19:24 supplier:platform:soc:power -> ../../../virtual/devlink/platform:soc:power--platform:3f980000.usb
-rw-r--r-- 1 root root 4096  1. Jan 1970  uevent
-r--r--r-- 1 root root 4096 17. Nov 19:24 waiting_for_supplier
root@kmxbilr2 /usr/local/sbin #

Kernel 5.4:

root@kmxbilr3p ~ # ll /sys/devices/platform/soc/3f980000.usb
insgesamt 0
-r--r--r-- 1 root root 4096 17. Nov 19:41 busconnected
-rw-r--r-- 1 root root 4096 17. Nov 19:25 buspower
-rw-r--r-- 1 root root 4096 17. Nov 19:41 bussuspend
-rw-r--r-- 1 root root 4096 17. Nov 19:41 devspeed
--w------- 1 root root 4096 17. Nov 19:41 disconnect_us
lrwxrwxrwx 1 root root    0 17. Nov 19:41 driver -> ../../../../bus/platform/drivers/dwc_otg
-rw-r--r-- 1 root root 4096 17. Nov 19:41 driver_override
-r--r--r-- 1 root root 4096 17. Nov 19:41 enumspeed
-rw-r--r-- 1 root root 4096 17. Nov 19:41 fr_interval
drwxr-xr-x 3 root root    0 17. Nov 19:34 gadget
-rw-r--r-- 1 root root 4096 17. Nov 19:41 ggpio
-rw-r--r-- 1 root root 4096 17. Nov 19:41 gnptxfsiz
-rw-r--r-- 1 root root 4096 17. Nov 19:41 gotgctl
-rw-r--r-- 1 root root 4096 17. Nov 19:41 gpvndctl
-rw-r--r-- 1 root root 4096 17. Nov 19:41 grxfsiz
-r--r--r-- 1 root root 4096 17. Nov 19:41 gsnpsid
-rw-r--r-- 1 root root 4096 17. Nov 19:41 guid
-rw-r--r-- 1 root root 4096 17. Nov 19:41 gusbcfg
-r--r--r-- 1 root root 4096 17. Nov 19:41 hcddump
-r--r--r-- 1 root root 4096 17. Nov 19:41 hcd_frrem
-rw-r--r-- 1 root root 4096 17. Nov 19:41 hnp
-rw-r--r-- 1 root root 4096 17. Nov 19:41 hnpcapable
-rw-r--r-- 1 root root 4096 17. Nov 19:41 hprt0
-r--r--r-- 1 root root 4096 17. Nov 19:41 hptxfsiz
-rw-r--r-- 1 root root 4096 17. Nov 19:41 hsic_connect
-rw-r--r-- 1 root root 4096 17. Nov 19:41 inv_sel_hsic
-r--r--r-- 1 root root 4096 17. Nov 19:41 modalias
-r--r--r-- 1 root root 4096 17. Nov 19:41 mode
-rw-r--r-- 1 root root 4096 17. Nov 19:41 mode_ch_tim_en
lrwxrwxrwx 1 root root    0 17. Nov 19:41 of_node -> ../../../../firmware/devicetree/base/soc/usb@7e980000
-r--r--r-- 1 root root 4096 17. Nov 19:41 pools
drwxr-xr-x 2 root root    0 17. Nov 19:34 power
-r--r--r-- 1 root root 4096 17. Nov 19:41 rd_reg_test
-r--r--r-- 1 root root 4096 17. Nov 19:41 regdump
-rw-r--r-- 1 root root 4096 17. Nov 19:41 regoffset
-rw-r--r-- 1 root root 4096 17. Nov 19:41 regvalue
-rw-r--r-- 1 root root 4096 17. Nov 19:41 remote_wakeup
-rw-r--r-- 1 root root 4096 17. Nov 19:41 rem_wakeup_pwrdn
-r--r--r-- 1 root root 4096 17. Nov 19:41 spramdump
-rw-r--r-- 1 root root 4096 17. Nov 19:41 srp
-rw-r--r-- 1 root root 4096 17. Nov 19:41 srpcapable
lrwxrwxrwx 1 root root    0 17. Nov 19:41 subsystem -> ../../../../bus/platform
-rw-r--r-- 1 root root 4096 17. Nov 19:41 uevent
drwxr-xr-x 6 root root    0 17. Nov 19:34 usb1
-r--r--r-- 1 root root 4096 17. Nov 19:41 wr_reg_test
root@kmxbilr3p ~ #

dmesg | grep dwc prints:

root@kmxbilr2 /usr/local/sbin # dmesg | grep dwc
[    3.280434] dwc_otg: version 3.00a 10-AUG-2012 (platform bus)
[    3.280670] dwc_otg: FIQ enabled
[    3.280684] dwc_otg: NAK holdoff enabled
[    3.280697] dwc_otg: FIQ split-transaction FSM enabled
[    3.280718] Module dwc_common_port init
root@kmxbilr2 /usr/local/sbin #

So I can assume that the USB driver is compiled into the kernel (as it says in the .config) and does something. Unfortunately the USB is completely dead

Oh yes, a

dtoverlay=dwc2,dr_mode=host

in /boot/config.txt did not bring me any further

So, what's going wrong again?

popcornmix commented 2 years ago

Are you hitting this? https://github.com/LibreELEC/LibreELEC.tv/pull/5863

mkreisl commented 2 years ago

@popcornmix Of course not, thanks for the hint

popcornmix commented 2 years ago

It's fine in rpi-update kernel. Ethernet, keyboard and mouse work.

pi@b3plus:~ $ uname -a
Linux b3plus 5.15.2-v7+ #1485 SMP Mon Nov 15 20:12:03 GMT 2021 armv7l GNU/Linux
pi@b3plus:~ $ lsusb
Bus 001 Device 007: ID 093a:2510 Pixart Imaging, Inc. Optical Mouse
Bus 001 Device 006: ID 04d9:0006 Holtek Semiconductor, Inc. Wired Keyboard (78/79 key) [RPI Wired Keyboard 5]
Bus 001 Device 004: ID 05e3:0610 Genesys Logic, Inc. Hub
Bus 001 Device 005: ID 0424:7800 Microchip Technology, Inc. (formerly SMSC) 
Bus 001 Device 003: ID 0424:2514 Microchip Technology, Inc. (formerly SMSC) USB 2.0 Hub
Bus 001 Device 002: ID 0424:2514 Microchip Technology, Inc. (formerly SMSC) USB 2.0 Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
mkreisl commented 2 years ago

With me the scrap unfortunately does not yet work as it should: Boot from the sd-card usb keyboard was there, ethernet did not work yet but was present but DOWN

Then boot from the network, doesn't work, then took again old firmware (the one from June) then it boots at least the kernel but again neither network nor keyboard there.

The whole thing really pisses me off again

pelwell commented 2 years ago

I think that waiting_for_supplier node is significant: https://mjmwired.net/kernel/Documentation/ABI/testing/sysfs-devices-waiting_for_supplier The kernel has detected that something on which the USB interface depends (a power domain, for example) is absent, and as a result isn't even trying to probe it. What does vcgencmd version say? My guess is that the firmware and the kernel are on opposite sides of the VEC clock change.

mkreisl commented 2 years ago

No, it has nothing to do with the firmware.

It was not enough to add these CONFIG parameters (which should be set automatically if done correctly), but for network boot the phy-generic module, which intelligently cannot be compiled into the kernel, must also be added to the initramfs. see https://github.com/xbianonpi/xbian-package-initramfs/commit/5a4b08e9ca216963d735a5b0af5272495797aca1

pelwell commented 2 years ago

The issue is exactly what @popcornmix suggested - that in newer kernels (rpi-5.14.y, rpi-5.15.y) you need to set CONFIG_NOP_USB_XCEIV=y to cause the phy-generic module to be built. This is done by all of the standard Raspberry Pi _defconfig files in rpi-5.14.y and later. If you are using different defconfigs then it is your responsibility to keep them up to date.

mkreisl commented 2 years ago

The build is NOT the problem, it's that it can't be compiled into the kernel. And of course this is especially good for network boot.

And no, you can't always start from scratch with a new kernel. you'll go crazy with the idiotic release changes every 3 months. A make olddefconfig should fix everything if everything is correct. But it often isn't.

pelwell commented 2 years ago

Does USB work on a Pi 3 using our standard kernel builds found in the next branch of our firmware repo (currently on 5.15.2)? Yes? Okay, now show me the generic PHY driver module.

Hint: There isn't one.

pelwell commented 2 years ago

I think your problem is explained by this comment in drivers/usb/phy/Kconfig:

config NOP_USB_XCEIV
    tristate "NOP USB Transceiver Driver"
    depends on USB_GADGET || !USB_GADGET # if USB_GADGET=m, NOP can't be built-in
    select USB_PHY
    help
      This driver is to be used by all the usb transceiver which are either
      built-in with usb ip or which are autonomous and doesn't require any
      phy programming such as ISP1x04 etc.

In rpi-5.10.y we have:

arch/arm/configs/bcm2709_defconfig:CONFIG_USB_GADGET=m
arch/arm/configs/bcm2711_defconfig:CONFIG_NOP_USB_XCEIV=y
arch/arm/configs/bcm2711_defconfig:CONFIG_USB_GADGET=y
arch/arm/configs/bcmrpi_defconfig:CONFIG_USB_GADGET=m
arch/arm64/configs/bcm2711_defconfig:CONFIG_USB_GADGET=y
arch/arm64/configs/bcm2711_defconfig:CONFIG_NOP_USB_XCEIV=y
arch/arm64/configs/bcmrpi3_defconfig:CONFIG_USB_GADGET=m

while in rpi-5.14.y there have been some changes:

arch/arm/configs/bcm2709_defconfig:CONFIG_NOP_USB_XCEIV=y
arch/arm/configs/bcm2709_defconfig:CONFIG_USB_GADGET=y
arch/arm/configs/bcm2711_defconfig:CONFIG_NOP_USB_XCEIV=y
arch/arm/configs/bcm2711_defconfig:CONFIG_USB_GADGET=y
arch/arm/configs/bcmrpi_defconfig:CONFIG_NOP_USB_XCEIV=y
arch/arm/configs/bcmrpi_defconfig:CONFIG_USB_GADGET=y
arch/arm64/configs/bcm2711_defconfig:CONFIG_NOP_USB_XCEIV=y
arch/arm64/configs/bcm2711_defconfig:CONFIG_USB_GADGET=y
arch/arm64/configs/bcmrpi3_defconfig:CONFIG_NOP_USB_XCEIV=y
arch/arm64/configs/bcmrpi3_defconfig:CONFIG_USB_GADGET=y

All platforms now build now request the generic PHY support, so all have to build in the USB gadget support.

The explanation can be found in this commit (51d4f324cef1c79df17ae3badc3645cf8bd4c1bd) , added in rpi-5.13.y:

commit 51d4f324cef1c79df17ae3badc3645cf8bd4c1bd
Author: Phil Elwell <phil@raspberrypi.com>
Date:   Mon Aug 2 14:48:41 2021 +0100

    configs: NOP_USB_XCEIV=y and USB_GADGET=y

    As of 5.13, "suppliers" (kinds of dependencies such as power and
    PHY drivers) are checked by the driver framework rather than the
    driver, making declaring such a supplier for a device without having
    a driver available effectively fatal even if the "consumer" driver
    makes no reference to it. The generic USB PHY declared for the DWC
    USB block is such a supplier, but the Pi 1-3 defconfigs don't include
    the generic PHY driver, making USB on those Pis unusable.

    Add USB generic PHY support to the remaining Pi defconfigs, which for
    Kconfig reasons also requires gadget support to be built-in (the kernel
    size increase appears to be minimal).

    See: https://github.com/raspberrypi/linux/issues/4496
mkreisl commented 2 years ago

@pelwell thanks for this info. I found this out myself last night after I calmed down a bit and analyzed the situation. The parameter CONFIG_USB_GADGET=m turned out to be the culprit.

pelwell commented 2 years ago

There is upstream churn that catches us out at most kernel version bumps, but hopefully by watching our patches and learning from our mistakes you can lessen the maintenance burden.

mkreisl commented 2 years ago

I can imagine that these version jumps are really annoying for you. Normally I always check if something has changed significantly in the defconfigs from time to time. But just with this issue I didn't do that, why I don't know anymore, because the error was already there for a long time and strangely didn't affect the Raspberry Pi 4.