Closed ieure closed 2 years ago
Unfortunately it's pretty hard for us to replicate your problem because we don't have the hardware, it might be worthwhile trying to bisect QMK and finding a release where it doesn't do this and figuring out exactly when it stopped working. Can you try a commit before 39bd760faf2666e91d6dc5b199f02fa3206c6acd for example?
There is an ongoing issue with Linux, QMK and ARM which we haven't been able to find a root cause for. Not too many of us have kinetis based boards, so I'm afraid we're going to have to rely on you to do some detective work.
Another thing which might be interesting to try is running Wireshark USB protocol analyzer and see if you can compare the trace against a working keyboard to see how they differ.
Have you tested other operating systems?
Interestingly I see that disconnect message with a razer mouse, but it's going through a bunch of different hubs.
Unfortunately it's pretty hard for us to replicate your problem because we don't have the hardware,
If someone wants to seriously look into this, I can loan them a keyboard which demonstrates the issue.
Can you try a commit before 39bd760 for example?
Sure, possibly in a few days.
Another thing which might be interesting to try is running Wireshark USB protocol analyzer and see if you can compare the trace against a working keyboard to see how they differ.
I'll look into this, though I don't really know anything about the USB protocol, so I'm not sure what I'd be looking for. I could provide the captures from my Ergodox and a random other USB keyboard. I think I have a HHKB with a hub, which might be a good candidate.
Have you tested other operating systems?
No, but I have Windows 10 and macOS boxes I could try. The keyboard works as I expect (mousekeys, too). But while it's plugged in, the kernel emits those messages. I'm not sure what the equivalent behavior would be on a different OS.
I have this same issue with an infinity ergodox I flashed QMK into. Keyb works fine, but floods the kernel with errors.
Could try a newer kernel. It appears that the linux kernel is extremely strict about it's processing of USB compared to other operating systems and they seem to have fixed something around the 5.2.9 time frame that was causing xinput to crash in older kernels. It's not entirely clear what changed.
@yanfali I've had this issue for months, just didn't bother reporting until now. I'm on kernel 5.3.2 right now, and I've had this problem since the early 4.x series at the very least.
Here's what dmesg looks like for me on an average day: https://gist.github.com/lovesegfault/1da5f3b8c5d6b8d800423cd25c5d953d
@lovesegfault this is powered by a kinetis ARM chip. QMK doesn't really have a lot of keyboards with this chipset, so it could be a bug in the hal for it. We haven't done a chibios update in a while. @Talljoe you appear to have one of these. Have you ever seen these errors if you use linux? @lovesegfault you appear to be using a docking station. Does it still happen when you use a port directly on the laptop?
@yanfali I can confirm that connecting the keyb straight to my laptop does not resolve the issue (exactly same message, on a different port)
@lovesegfault do you have any other linux computers you can try? Do you have any other keyboards running QMK ? Just trying to isolate if it's this computer or this particular keyboard.
@yanfali I've tried other Linuxes and the issue persists, IIRC. I can test again tomorrow at work on my coworkers' boxes.
I have another QMK keyboard, but that works over USB-C (UT-47.2). I do not have the same issues there.
@lovesegfault thanks. Can you disable CONSOLE ? We're getting reports that Linux doesn't understand what to do with it and it may be causing errors. More precisely, the format of the HID reporting from ARM appears to be causing errors on Linux systems.
@yanfali
Can you disable CONSOLE ?
How?
You have to build from source, but edit rules.mk for your keyboard and set console to no.
On Wed, Oct 2, 2019, 13:11 Bernardo Meurer notifications@github.com wrote:
@yanfali https://github.com/yanfali
Can you disable CONSOLE ?
How?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/qmk/qmk_firmware/issues/6713?email_source=notifications&email_token=AARLSU6C4NEHMHEDFAO2G33QMT57NA5CNFSM4IVKCY4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAGATLA#issuecomment-537659820, or mute the thread https://github.com/notifications/unsubscribe-auth/AARLSUYLLYCLWNS34GTSNC3QMT57NANCNFSM4IVKCY4A .
Would it be possible for ChibiOS to be updated so I can test it?
This is the latest release: http://www.chibios.com/forum/viewtopic.php?f=7&t=4979&p=35807#p35807
The current SVN mirror used by QMK is unmaintained.
It's a pretty big task, so no one has stepped up. @e11i0t23 was doing something but got busy with school.
I rebuilt my firmware with CONSOLE
disabled and early signs are promising. It's been running for a few days and I have a few of the "Cannot enable. Maybe the USB cable is bad?" messages -- but not tens of thousands.
I also noticed that if you ran sudo lshw
with the bad firmware, it would immediately trigger the issue -- lshw
would never complete and the kernel would start spewing lots. I'm not seeing that behavior with CONSOLE
disabled now.
QMK now has CONSOLE_ENABLE=no
by default for the ergodox_infinity
but I can still repro the behavior, unfortunately.
[ 2204.804771] usb 6-2-port3: Cannot enable. Maybe the USB cable is bad?
[ 2208.860910] usb 6-2-port3: Cannot enable. Maybe the USB cable is bad?
[ 2212.917091] usb 6-2-port3: Cannot enable. Maybe the USB cable is bad?
[ 2216.973028] usb 6-2-port3: Cannot enable. Maybe the USB cable is bad?
[ 2221.029053] usb 6-2-port3: Cannot enable. Maybe the USB cable is bad?
[ 2225.085108] usb 6-2-port3: Cannot enable. Maybe the USB cable is bad?
I just recently noticed this issue with my Ergodox Infinity and found a fix that has solved the issue for me:
The problem is that the extra USB port (the empty USB-A port in the side of the ergodox which is not plugged into the computer) is meant to supply power only but the kernel is trying to make it function as a USB hub.
The output of dmesg -w
for my case was a lot of
[ 872.816596] usb 8-2-port2: Cannot enable. Maybe the USB cable is bad?
[ 876.860081] usb 8-2-port2: Cannot enable. Maybe the USB cable is bad?
[ 880.899741] usb 8-2-port2: Cannot enable. Maybe the USB cable is bad?
and the relevant output of lsusb -t
/: Bus 08.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 10000M
|__ Port 2: Dev 2, If 0, Class=Hub, Driver=hub/4p, 10000M
|__ Port 1: Dev 3, If 0, Class=Hub, Driver=hub/4p, 5000M
|__ Port 2: Dev 4, If 0, Class=Vendor Specific Class, Driver=r8152, 5000M
So running
echo '8-2' | sudo tee /sys/bus/usb/drivers/usb/unbind
disabled the problematic port altogether and made the errors stop, while not affecting any functionality. The port still allows powering of other devices as usual. This isn't an ideal fix since it requires some steps and would need to be changed if the keyboard changes ports or if the ports change names (which I don't believe should happen since the error was the same across reboots for me), but it will stop the flood of dmesg errors. EDIT: It seems that this fix is very temporary as the issue comes back after a reboot.
Thanks @burrhole. I have also been having this issue with my Infinity Ergodox. In my case the error message from the kernel has been:
usb usb4-port2: Cannot enable. Maybe the USB cable is bad?
So my command to disable that port looked like this:
echo 'usb4' | sudo tee /sys/bus/usb/drivers/usb/unbind
I guess I could put this command into a systemd service file and have it run on boot, but like you said that would break if I ever swap ports or the ports change names or anything along those lines.
I only started seeing this issue after I ran an update recently. There were nearly a thousand packages in the update, so it would be impractical for me to isolate just one package that triggered the issue, but FWIW the kernel was upgraded 5.6.15 -> 5.9.10. For (hopefully) obvious reasons I'm not going to roll back the kernel package to test if that was the issue.
Does #14814 have any effect on this? Alternatively, would you be able to do a Wireshark USB capture while this is happening? (please be aware Wireshark will capture anything you type with your keyboard while recording)
I had been noticing this on my Ubuntu 18.04 desktop for some time. Hasn't been much of a problem until now. Would only appear a half dozen times & then the boot would resume. Now it's sometimes so bad that it just loops & fills the screen. I have had to shut down & retry. I have even noticed the same error showing up a few times during shutdown. I'm pretty much a Linux newbie, so I can only hope there is something without a lot of command line work. All of my USB ports are working, so I have no idea what this means.
This issue has been automatically closed because it has not had any recent activity. If this issue is still valid, re-open the issue and let us know.
This is still a problem at least for me on Ergodox Infinity. Also when I boot the computer with the keyboard plugged-in it isn't detected, I always have to manually reconnect the cable (might be the same as https://github.com/qmk/qmk_firmware/issues/18759). This happens with the default config and keymap.
Also I just applied #14814 to latest master and it does not solve the issue for me.
Encountered this on a brand new box with ASUS Prime B550-Plus AC-HES motherboard. The actual report was for usb6-port1; "lsusb -tv" identified as "ID 1d6b:0003 Linux Foundation 3.0 root hub" with nothing else chained off it. The workaround above for disabling usb6 worked fine. Tracked the problem down to plugging in a chain of junky USB 2 stuff (two hubs chained plus an empty hub plus a webcam and headphones; don't ask) plugged into a USB 3 port on the case. All the "stuff" worked fine, but the system ran slow and logged messages every few seconds. Plugging the chain of junk into a USB 2 port on the case made the problem go away. If I can do anything to help identify/troubleshoot further, lemme know.
I confronted the same issue when I attempted to connect via USB-to-USB a Linux computer with a Raspberry Pi. My solution is based on the suggested comments above: Basically, you need to unbind
and then bind
this port by executing the following commands:
echo 'usb2’ | sudo tee /sys/bus/usb/drivers/usb/unbind
echo 'usb2’ | sudo tee /sys/bus/usb/drivers/usb/bind
And then by executing the simple command lsusb
your device will be listed
You may apply this approach to any USB-to-USB device connection. Hope this helps.
I ran into this issue as well with my Ergodox Infinity. It seemed like it only occurs when plugged into a USB3 port.
I was getting these messages in dmesg:
[ 1193.989950] usb 6-3-port2: Cannot enable. Maybe the USB cable is bad?
[ 1198.029671] usb 6-3-port2: Cannot enable. Maybe the USB cable is bad?
[ 1202.069433] usb 6-3-port2: Cannot enable. Maybe the USB cable is bad?
[ 1206.109194] usb 6-3-port2: Cannot enable. Maybe the USB cable is bad?
[ 1210.148960] usb 6-3-port2: Cannot enable. Maybe the USB cable is bad?
[ 1214.188718] usb 6-3-port2: Cannot enable. Maybe the USB cable is bad?
[ 1218.228489] usb 6-3-port2: Cannot enable. Maybe the USB cable is bad?
[ 1222.268235] usb 6-3-port2: Cannot enable. Maybe the USB cable is bad?
...but to my surprise, the Ergodox was actually on an adjacent bus:
❯ lsusb -t -v
/: Bus 06.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 10000M
ID 1d6b:0003 Linux Foundation 3.0 root hub
|__ Port 2: Dev 2, If 0, Class=Hub, Driver=hub/4p, 5000M
ID 2109:0812 VIA Labs, Inc. VL812 Hub
|__ Port 4: Dev 4, If 0, Class=Hub, Driver=hub/4p, 5000M
ID 2109:0812 VIA Labs, Inc. VL812 Hub
|__ Port 4: Dev 6, If 0, Class=Hub, Driver=hub/4p, 5000M
ID 2109:0812 VIA Labs, Inc. VL812 Hub
|__ Port 3: Dev 3, If 0, Class=Hub, Driver=hub/4p, 5000M
ID 0b00:0483 INGENICO
|__ Port 1: Dev 5, If 0, Class=Hub, Driver=hub/4p, 5000M
ID 0bda:0483 Realtek Semiconductor Corp.
|__ Port 2: Dev 7, If 0, Class=Communications, Driver=, 5000M
ID 0bda:8153 Realtek Semiconductor Corp. RTL8153 Gigabit Ethernet Adapter
|__ Port 2: Dev 7, If 1, Class=CDC Data, Driver=, 5000M
ID 0bda:8153 Realtek Semiconductor Corp. RTL8153 Gigabit Ethernet Adapter
|__ Port 3: Dev 8, If 0, Class=Mass Storage, Driver=usb-storage, 5000M
ID 0bda:0306 Realtek Semiconductor Corp.
/: Bus 05.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 480M
ID 1d6b:0002 Linux Foundation 2.0 root hub
|__ Port 2: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
ID 2109:2812 VIA Labs, Inc. VL812 Hub
|__ Port 1: Dev 4, If 0, Class=Audio, Driver=snd-usb-audio, 12M
ID 08bb:2902 Texas Instruments PCM2902 Audio Codec
|__ Port 1: Dev 4, If 1, Class=Audio, Driver=snd-usb-audio, 12M
ID 08bb:2902 Texas Instruments PCM2902 Audio Codec
|__ Port 1: Dev 4, If 2, Class=Audio, Driver=snd-usb-audio, 12M
ID 08bb:2902 Texas Instruments PCM2902 Audio Codec
|__ Port 1: Dev 4, If 3, Class=Human Interface Device, Driver=usbhid, 12M
ID 08bb:2902 Texas Instruments PCM2902 Audio Codec
|__ Port 4: Dev 5, If 0, Class=Hub, Driver=hub/4p, 480M
ID 2109:2812 VIA Labs, Inc. VL812 Hub
|__ Port 4: Dev 7, If 0, Class=Hub, Driver=hub/4p, 480M
ID 2109:2812 VIA Labs, Inc. VL812 Hub
|__ Port 3: Dev 3, If 0, Class=Hub, Driver=hub/5p, 480M
ID 0b00:5483 INGENICO
|__ Port 1: Dev 6, If 0, Class=Hub, Driver=hub/5p, 480M
ID 0bda:5483 Realtek Semiconductor Corp.
|__ Port 2: Dev 8, If 0, Class=Human Interface Device, Driver=usbhid, 12M
ID 1c11:b04d Input Club Inc. ErgoDox Infinity
|__ Port 2: Dev 8, If 1, Class=Human Interface Device, Driver=usbhid, 12M
ID 1c11:b04d Input Club Inc. ErgoDox Infinity
|__ Port 3: Dev 9, If 0, Class=Human Interface Device, Driver=usbhid, 12M
ID 046d:c547 Logitech, Inc.
|__ Port 3: Dev 9, If 1, Class=Human Interface Device, Driver=usbhid, 12M
ID 046d:c547 Logitech, Inc.
|__ Port 3: Dev 9, If 2, Class=Human Interface Device, Driver=usbhid, 12M
ID 046d:c547 Logitech, Inc.
|__ Port 5: Dev 10, If 0, Class=Human Interface Device, Driver=usbhid, 480M
ID 0bda:1100 Realtek Semiconductor Corp.
...
udev info:
❯ sudo udevadm info -p /bus/usb/devices/6-3
P: /devices/pci0000:00/0000:00:08.1/0000:10:00.3/usb6/6-3
M: 6-3
R: 3
U: usb
T: usb_device
D: c 189:642
N: bus/usb/006/003
L: 0
E: DEVPATH=/devices/pci0000:00/0000:00:08.1/0000:10:00.3/usb6/6-3
E: SUBSYSTEM=usb
E: DEVNAME=/dev/bus/usb/006/003
E: DEVTYPE=usb_device
E: PRODUCT=b00/483/4
E: TYPE=9/0/3
E: BUSNUM=006
E: DEVNUM=003
E: MAJOR=189
E: MINOR=642
E: USEC_INITIALIZED=70100211
E: ID_BUS=usb
E: ID_MODEL=4-Port_USB_3.0_Hub
E: ID_MODEL_ENC=4-Port\x20USB\x203.0\x20Hub
E: ID_MODEL_ID=0483
E: ID_SERIAL=Generic_4-Port_USB_3.0_Hub
E: ID_VENDOR=Generic
E: ID_VENDOR_ENC=Generic
E: ID_VENDOR_ID=0b00
E: ID_REVISION=0004
E: ID_USB_MODEL=4-Port_USB_3.0_Hub
E: ID_USB_MODEL_ENC=4-Port\x20USB\x203.0\x20Hub
E: ID_USB_MODEL_ID=0483
E: ID_USB_SERIAL=Generic_4-Port_USB_3.0_Hub
E: ID_USB_VENDOR=Generic
E: ID_USB_VENDOR_ENC=Generic
E: ID_USB_VENDOR_ID=0b00
E: ID_USB_REVISION=0004
E: ID_USB_INTERFACES=:090000:
E: ID_VENDOR_FROM_DATABASE=INGENICO
E: ID_PATH=pci-0000:10:00.3-usb-0:3
E: ID_PATH_TAG=pci-0000_10_00_3-usb-0_3
E: ID_FOR_SEAT=usb-pci-0000_10_00_3-usb-0_3
E: TAGS=:seat:
E: CURRENT_TAGS=:seat:
For now, I just have it unbound with a systemd unit:
❯ cat /etc/systemd/system/disable-6-3.service
[Unit]
Description=Turn off bad USB port
[Service]
Type=oneshot
ExecStart=/bin/sh -c 'echo 6-3 > /sys/bus/usb/drivers/usb/unbind'
[Install]
WantedBy=multi-user.target
It appears that unauthorizing or disabling the USB device via udev isn't enough, and the port itself needs to be unbound.
I still think the Ergodox itself is causing this somehow, but debugging the issue seems like it'll be pretty involved.
Apparently Ergodox Infinity abuses the USB3 data lines of its Type-C port for a serial port connection:
This is not the intended way to use the USB3 data lines, and maybe with some USB hosts or hubs the USB3 attach detection process gives a wrong result due to the presence of that connection; maybe some implementation end up in a continuous “new device detected but not really responding” state. It is also possible that some different QMK firmware versions for Ergodox Infinity initialized those serial port pins in a different way, and that affects the USB3 attach detection somehow.
Most likely the problem won't appear if you use some USB2-only cable (without any USB3 data lines) to connect the keyboard.
Started getting caught in this loop. Kept getting worse. Finally after booting up my Ubuntu was running very slow. My son told me to replace the motherboard. That fixed it. Nothing wrong with my system.
Apparently Ergodox Infinity abuses the USB3 data lines of its Type-C port for a serial port connection:
This is for the communication between the two halves of the keyboard, and should have nothing to do at all with the computer-to-keyboard side of things.
This is for the communication between the two halves of the keyboard, and should have nothing to do at all with the computer-to-keyboard side of things.
Yes, the USB3 data lines in the other pair of ports are actually used for serial communication between halves; however, the master USB port still has those lines connected to the serial port pins on the MCU, even though those pins are not actually enabled as serial port pins. Not sure what state they would end up in though — the firmware does not seem to initialize the state of those pins except in usart_master_init()
and usart_slave_init()
, so E0
and E1
won't be initialized on the master side; they would probably keep the configuration as set in platforms/chibios/boards/IC_TEENSY_3_1/board/board.c
, which is apparently PAL_MODE_OUTPUT_PUSHPULL
. The problem here is that apparently the USB3 host detects whether there is any USB3 device connected by measuring the impedance of those data lines, and that measurement might give a false positive if the data lines are connected to some MCU pins instead of leaving them floating.
@sigprof that's a very good point! I forgot that the halves are symmetrical and the USB ssrx pins are connected on the host half as well. I'll try and see if I can set those pins to hi-Z.
Tried setting PORTE PCR MUX register to 0 (analog) in usart_master_init()
, and the errors have gone away! I'll try it out for a bit and see if I run into any issues.
@Hylian can you share a diff? I'd be willing to test this as well.
Sure, you can find the patch here: https://github.com/Hylian/qmk_firmware/commit/996d4393fd7d72649c337a2ccb6487a53bc50a45
So far, it seems like the the bad USB cable message has been entirely fixed with that patch. However, the right-half flakiness that I was investigating in #19420 still persists. This is on top of the commit prior to the bad commit that we found in that issue. Subjectively, it seems like the right half is hanging more frequently while in-use now, while previously, it was hanging more during boot/shutdown. Hard to tell if it's related to this fix at all. The QMK build from 2021 I was using previously didn't have any such issues, but I'd like to avoid a bisect of two years worth of commits for an intermittent issue if possible...
One theory could be that upon losing serial connection, the slave-half goes into usart master mode, and muxes the usart pins to hi-Z, requiring a power cycle. I haven't dug into the architecture of the master/slave detection enough to know what codepaths could be running, though.
One theory could be that upon losing serial connection, the slave-half goes into usart master mode, and muxes the usart pins to hi-Z, requiring a power cycle.
This should not happen — QMK selects the role only during startup and then never switches it (the SPLIT_WATCHDOG_ENABLE
code reboots the MCU if the slave half did not hear anything from master, so that it could come up again as master in case of misdetection). And the master detection happens after the USB host completes the enumeration process (after SET_CONFIGURATION
), so it's probably impossible to get a false positive (a false negative is much more likely, which prompted the development of the split watchdog feature).
Gotcha, thanks for the background! Do you have any tips on getting any debug output from the slave half? Is WDT enabled by default? ie. Right half appears to hang because of loss of UART and not an actual hang.
I could try to get some logic analyzer captures on the UART pins, but I doubt it would be that helpful tbh. Maybe I could output some continuously updating information to the LCD screen to see if tasks are being serviced.
You can try adding some custom code to output debug info over a spare UART, or connect a hardware debugger over SWD (the PCB apparently has breakout pads for SWD and all UARTs, including one that is not used by the firmware). Or even hijack the UART that is normally used by the master side and accessible over the USB-A port pins.
Just now I had the same problem and I figure it out. Some of my hardware are b450m(MSI)+2400G+monitor(VX2880-4K-HDU-2)+mouse(G304)+keyboard(Ikbc C104). My linux environment is debian 11+kenel(5.10.0.23)+kde(plasma 5.20.5). When I plug usb cable to monitor above and plug mouse/keyboard to the monitor usb hub, the "usb port cannot enable" problem exists. Then I remove the cable and plug mouse/keyboard directly to two mainboard usb ports seperately , this problem disappeares. Hope to be helpful.
Describe the Bug
After some amount of time having my Infinity Ergodox plugged in, my Linux kernel outputs a message every four seconds:
Unplugging and replugging the keyboard stops the messages, but they return after some time.
I've reproduced this on multiple machines, multiple keyboards, and multiple kernel versions.
On my computer that's powered on 24/7, this produces ~57,000 log messages per day, or around 3x the volume of all other kernel log messages combined.
System Information
avr-gcc (GCC) 5.4.0
(buster); also 4.9.2 (stretch)arm-none-eabi-gcc (15:7-2018-q2-6) 7.3.1 20180622 (release) [ARM/embedded-7-branch revision 261907]
(buster); also 5.4.1 (stretch)Additional Context
I have verified this issue on the following machines and configurations:
Here's my specific keyboard configuration. I also have the exact blobs I flashed, if necessary.
If I run
lsusb -v
, it hangs, and the kernel produces these messages:I strongly suspect that this is related to the mouse keys support, which I have enabled.