[Bug] KB2040 not detected after booting through GRUB

subrezon commented 1 year ago

Describe the Bug

I have a Sofle split keyboard with a KB2040 on each side, connected to a dual-boot PC with Ubuntu 22.04 and Windows 11. It is connected to a USB hub in my monitor (Dell S2721DGFA), which is connected to the PC. I boot using GRUB, then choose the OS I want to boot to.

When I turn my PC on I can use the keyboard in GRUB to choose the OS, but after booting into the OS the keyboard does not get detected until I unplug it and plug it back in.

Before unplugging, Sofle keyboard clearly missing:

 subrezon@bubuntu  ~  lsusb
Bus 004 Device 002: ID 174c:225c ASMedia Technology Inc. Ugreen Storage Device
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 007: ID 1462:7c56 Micro Star International MYSTIC LIGHT 
Bus 001 Device 011: ID 1532:007e Razer USA, Ltd RC30-030502 Mouse Dock
Bus 001 Device 009: ID 1532:007b Razer USA, Ltd RC30-0305 Gaming Mouse Dongle [Viper Ultimate (Wireless)]
Bus 001 Device 006: ID 05e3:0610 Genesys Logic, Inc. Hub
Bus 001 Device 013: ID 0424:274b Microchip Technology, Inc. (formerly SMSC) Hub Controller
Bus 001 Device 012: ID 056e:0132 Elecom Co., Ltd TrackBall Mouse M-DPT1MR Wireless
Bus 001 Device 010: ID 046d:082d Logitech, Inc. HD Pro Webcam C920
Bus 001 Device 008: ID 2708:0009 Audient Audient iD4
Bus 001 Device 005: ID 0424:2734 Microchip Technology, Inc. (formerly SMSC) USB2734
Bus 001 Device 003: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 004: ID 8087:0032 Intel Corp. AX210 Bluetooth
Bus 001 Device 002: ID 05e3:0608 Genesys Logic, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

After unplugging and plugging back in, keyboard is detected and works:

 subrezon@bubuntu  ~  lsusb
Bus 004 Device 002: ID 174c:225c ASMedia Technology Inc. Ugreen Storage Device
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 007: ID 1462:7c56 Micro Star International MYSTIC LIGHT 
Bus 001 Device 011: ID 1532:007e Razer USA, Ltd RC30-030502 Mouse Dock
Bus 001 Device 009: ID 1532:007b Razer USA, Ltd RC30-0305 Gaming Mouse Dongle [Viper Ultimate (Wireless)]
Bus 001 Device 006: ID 05e3:0610 Genesys Logic, Inc. Hub
Bus 001 Device 013: ID 0424:274b Microchip Technology, Inc. (formerly SMSC) Hub Controller
Bus 001 Device 012: ID 056e:0132 Elecom Co., Ltd TrackBall Mouse M-DPT1MR Wireless
Bus 001 Device 014: ID fc32:0287 JosefAdamcik Sofle
Bus 001 Device 010: ID 046d:082d Logitech, Inc. HD Pro Webcam C920
Bus 001 Device 008: ID 2708:0009 Audient Audient iD4
Bus 001 Device 005: ID 0424:2734 Microchip Technology, Inc. (formerly SMSC) USB2734
Bus 001 Device 003: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 004: ID 8087:0032 Intel Corp. AX210 Bluetooth
Bus 001 Device 002: ID 05e3:0608 Genesys Logic, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Keyboard Used

sofle (with CONVERT_TO=kb2040)

Link to product page (if applicable)

No response

Operating System

Ubuntu 22.04 / Windows

qmk doctor Output

subrezon@bubuntu  ~/Keyboards/qmk/qmk_firmware   subrezon--master ±  qmk doctor
Ψ QMK Doctor is checking your environment.
Ψ CLI version: 1.1.2
Ψ QMK home: /home/subrezon/Keyboards/qmk/qmk_firmware
Ψ Detected Linux (Ubuntu 22.04.2 LTS).
Ψ Git branch: subrezon--master
Ψ Repo version: 0.16.9
⚠ Git has unstashed/uncommitted changes.
Ψ - Latest subrezon--master: 2023-06-09 20:16:51 +0200 (d24474b061) -- remove umlaut layer + 200 ms tapping
Ψ - Latest upstream/master: 2023-06-09 16:51:17 +0200 (04b7a262e0) -- Fix typo (#21189)
Ψ - Latest upstream/develop: None
Ψ - Common ancestor with upstream/master: 2023-06-02 08:45:54 +0100 (ae5bcaac0d) -- [keyboard] Phoenix (#21051)
Ψ - Common ancestor with upstream/develop: None
Ψ All dependencies are installed.
Ψ Found arm-none-eabi-gcc version 10.3.1
Ψ Found avr-gcc version 5.4.0
Ψ Found avrdude version 6.3-20171130
Ψ Found dfu-programmer version 0.6.1
Ψ Found dfu-util version 0.9
Ψ Submodules are up to date.
Ψ Submodule status:
Ψ - lib/chibios: 2023-04-15 13:48:04 +0000 --  (11edb1610)
Ψ - lib/chibios-contrib: 2023-01-11 16:42:27 +0100 --  (a224be15)
Ψ - lib/googletest: 2021-06-11 06:37:43 -0700 --  (e2239ee6)
Ψ - lib/lufa: 2022-08-26 12:09:55 +1000 --  (549b97320)
Ψ - lib/vusb: 2022-06-13 09:18:17 +1000 --  (819dbc1)
Ψ - lib/printf: 2022-06-29 23:59:58 +0300 --  (c2e3b4e)
Ψ - lib/pico-sdk: 2023-02-12 20:19:37 +0100 --  (a3398d8)
Ψ - lib/lvgl: 2022-04-11 04:44:53 -0600 --  (e19410f8)
Ψ QMK is ready to go, but minor problems were found

Is AutoHotKey / Karabiner installed

[ ] AutoHotKey (Windows)
[ ] Karabiner (macOS)

Other keyboard-related software installed

none

Additional Context

Debugging attempted so far:

Using KMK on the keyboard DOES NOT result in this behavior. It seems to be exclusive to QMK.
Using atmega32u4 Pro Micros DOES NOT result in this behavior. This issue has been (so far) exclusive to KB2040.
Plugging USB into the other half DOES result in this behavior. Both KB2040s having the same hardware issue is highly unlikely.
Booting into Arch installation media or into Windows 11 through GRUB chainloading DOES result in this behavior. This issue is not exclusive to Ubuntu, or Linux in general.
Booting into Windows 11 directly without GRUB chainloading DOES NOT result in this behavior. The issue seems to be triggered by GRUB only (so far).
This issue is also present on my work laptop, also running Ubuntu 22.04. So the issue is not exclusive to a single PC.
Plugging the keyboard directly into the system (without a USB hub) results in the keyboard not working even in GRUB, when using a hub it at least worked in GRUB.

dexter93 commented 1 year ago

GRUB used to be problematic when using nkro, maybe it's still the case. see https://github.com/kiibohd/configurator/issues/134

subrezon commented 1 year ago

I only have 6KRO enabled.

subrezon commented 1 year ago

BIOS also seems to trigger this behavior. If I turn my PC on, press Delete to get into BIOS, and exit BIOS without rebooting, keyboard is not detected even when booting into Windows without GRUB.

Using KMK on the same board or AVR Pro Micros does not result in the same issue. Is there any way I can extract some debugging information out of the keyboard while the PC is booting?

KarlK90 commented 1 year ago

@subrezon there have been several fixes to the USB driver of the RP2040 which are currently only available on develop, could you please rebase to develop and see if this has been fixed?

subrezon commented 1 year ago

@KarlK90 cloned official repo, checked out develop and built my keymap using that. The issue seems to still occur, I will do a more detailed test later.

subrezon commented 1 year ago

@KarlK90 did more extensive testing using the develop branch, and it does indeed fix some of the issues. When connecting directly to the host without using a USB hub, the issue does not occur in any scenario - booting through GRUB into both Linux and Windows does not require replugging the keyboard anymore.

However, when connecting through the USB hub everything is still the exact same.

KarlK90 commented 1 year ago

Okay so some improvements, that is good news. As you are using a split keyboard and the kb2040 doesn't support vbus detection by design, qmk relies on a successful USB enumeration (basically the keyboard waits some seconds if it gets a connection to the host) to determine if the side is plugged into the host or is acting as the peripheral side. I suspect that this might fail for some reason when connected to the USB hub. There is an alternate mechanism which can be enabled by defining #define SPLIT_WATCHDOG_ENABLE in your board config.h. Could you try this? Please note that both sides have to be flashed.

subrezon commented 1 year ago

I suspect this might be due to the hub being always powered - the power LEDs of the KB2040s are always on, even when the PC is off. Something times out and then when the enumeration is supposed to happen - it doesn't.

I just did some quick testing with #define SPLIT_WATCHDOG_ENABLE on my work laptop (which the keyboard has the same issue with) and could successfully use the board both during and after GRUB. I will do the full test suite later today, and I also grabbed several other USB hubs to test with. I'll post an update in like ~4h.

subrezon commented 1 year ago

Tested #define SPLIT_WATCHDOG_ENABLE, everything works perfectly. Are there any drawbacks to having it enabled?

KarlK90 commented 1 year ago

Nice! Problem solved. There are no drawbacks. Enjoy your keyboard.

subrezon commented 1 year ago

Cool, thanks! Maybe I'll look into contributing something into the docs so it's easier to find and solve on one's own in the future...

qmk / qmk_firmware