raspberrypi / linux

Kernel source tree for Raspberry Pi-provided kernel builds. Issues unrelated to the linux kernel should be posted on the community forum at https://forums.raspberrypi.com/
Other
11.16k stars 5k forks source link

[USB] Broken pipe while enumerating Hub+LAN dongle #2734

Closed FD- closed 5 years ago

FD- commented 6 years ago

I'm having troubles getting a Hub+Ethernet dongle to work with the Raspberry Pi. Whenever I attach the dongle to the Pi, the integrated ethernet chip doesn't show in lsusb. The kernel log contains the information that it could not be enumerated:

[   68.131540] usb 1-1.3: new high-speed USB device number 8 using dwc_otg
[   68.262202] usb 1-1.3: New USB device found, idVendor=1a40, idProduct=0101
[   68.262217] usb 1-1.3: New USB device strings: Mfr=0, Product=1, SerialNumber=0
[   68.262225] usb 1-1.3: Product: USB2.0 HUB
[   68.265403] hub 1-1.3:1.0: USB hub found
[   68.266375] hub 1-1.3:1.0: 4 ports detected
[   68.581541] usb 1-1.3.2: new full-speed USB device number 9 using dwc_otg
[   68.681553] usb 1-1.3.2: device descriptor read/64, error -32
[   68.901535] usb 1-1.3.2: device descriptor read/64, error -32
[   69.121531] usb 1-1.3.2: new full-speed USB device number 10 using dwc_otg
[   69.221528] usb 1-1.3.2: device descriptor read/64, error -32
[   69.441534] usb 1-1.3.2: device descriptor read/64, error -32
[   69.561630] usb 1-1.3-port2: attempt power cycle
[   70.221589] usb 1-1.3.2: new full-speed USB device number 11 using dwc_otg
[   70.661591] usb 1-1.3.2: device not accepting address 11, error -32
[   70.761596] usb 1-1.3.2: new full-speed USB device number 12 using dwc_otg
[   71.201611] usb 1-1.3.2: device not accepting address 12, error -32
[   71.201718] usb 1-1.3-port2: unable to enumerate USB device
Full dmesg (click to expand)

``` [ 0.000000] Booting Linux on physical CPU 0x0 [ 0.000000] Linux version 4.14.78-v7+ (dc4@dc4-XPS13-9333) (gcc version 4.9.3 (crosstool-NG crosstool-ng-1.22.0-88-g8460611)) #1156 SMP Tue Oct 23 14:34:39 BST 2018 [ 0.000000] CPU: ARMv7 Processor [410fd034] revision 4 (ARMv7), cr=10c5383d [ 0.000000] CPU: div instructions available: patching division code [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache [ 0.000000] OF: fdt: Machine model: Raspberry Pi 3 Model B Plus Rev 1.3 [ 0.000000] Memory policy: Data cache writealloc [ 0.000000] cma: Reserved 8 MiB at 0x3ac00000 [ 0.000000] On node 0 totalpages: 242688 [ 0.000000] free_area_init_node: node 0, pgdat 80c85280, node_mem_map ba39f000 [ 0.000000] Normal zone: 2133 pages used for memmap [ 0.000000] Normal zone: 0 pages reserved [ 0.000000] Normal zone: 242688 pages, LIFO batch:31 [ 0.000000] percpu: Embedded 17 pages/cpu @ba348000 s38720 r8192 d22720 u69632 [ 0.000000] pcpu-alloc: s38720 r8192 d22720 u69632 alloc=17*4096 [ 0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 240555 [ 0.000000] Kernel command line: 8250.nr_uarts=0 bcm2708_fb.fbwidth=656 bcm2708_fb.fbheight=416 bcm2708_fb.fbswap=1 vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000 dwc_otg.lpm_enable=0 console=ttyS0,115200 console=tty1 root=PARTUUID=0bd3cdbf-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet splash plymouth.ignore-serial-consoles [ 0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes) [ 0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes) [ 0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes) [ 0.000000] Memory: 940232K/970752K available (7168K kernel code, 576K rwdata, 2076K rodata, 1024K init, 698K bss, 22328K reserved, 8192K cma-reserved) [ 0.000000] Virtual kernel memory layout: vector : 0xffff0000 - 0xffff1000 ( 4 kB) fixmap : 0xffc00000 - 0xfff00000 (3072 kB) vmalloc : 0xbb800000 - 0xff800000 (1088 MB) lowmem : 0x80000000 - 0xbb400000 ( 948 MB) modules : 0x7f000000 - 0x80000000 ( 16 MB) .text : 0x80008000 - 0x80800000 (8160 kB) .init : 0x80b00000 - 0x80c00000 (1024 kB) .data : 0x80c00000 - 0x80c9017c ( 577 kB) .bss : 0x80c97f04 - 0x80d468b0 ( 699 kB) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] ftrace: allocating 25284 entries in 75 pages [ 0.000000] Hierarchical RCU implementation. [ 0.000000] NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16 [ 0.000000] arch_timer: cp15 timer(s) running at 19.20MHz (phys). [ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x46d987e47, max_idle_ns: 440795202767 ns [ 0.000007] sched_clock: 56 bits at 19MHz, resolution 52ns, wraps every 4398046511078ns [ 0.000019] Switching to timer-based delay loop, resolution 52ns [ 0.000267] Console: colour dummy device 80x30 [ 0.000284] console [tty1] enabled [ 0.000309] Calibrating delay loop (skipped), value calculated using timer frequency.. 38.40 BogoMIPS (lpj=192000) [ 0.000324] pid_max: default: 32768 minimum: 301 [ 0.000647] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes) [ 0.000662] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes) [ 0.001597] Disabling memory control group subsystem [ 0.001675] CPU: Testing write buffer coherency: ok [ 0.002091] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000 [ 0.002490] Setting up static identity map for 0x100000 - 0x10003c [ 0.002611] Hierarchical SRCU implementation. [ 0.003285] smp: Bringing up secondary CPUs ... [ 0.004060] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001 [ 0.004899] CPU2: thread -1, cpu 2, socket 0, mpidr 80000002 [ 0.005722] CPU3: thread -1, cpu 3, socket 0, mpidr 80000003 [ 0.005828] smp: Brought up 1 node, 4 CPUs [ 0.005839] SMP: Total of 4 processors activated (153.60 BogoMIPS). [ 0.005844] CPU: All CPU(s) started in HYP mode. [ 0.005848] CPU: Virtualization extensions available. [ 0.006768] devtmpfs: initialized [ 0.016967] random: get_random_u32 called from bucket_table_alloc+0xfc/0x24c with crng_init=0 [ 0.017735] VFP support v0.3: implementor 41 architecture 3 part 40 variant 3 rev 4 [ 0.017958] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns [ 0.017975] futex hash table entries: 1024 (order: 4, 65536 bytes) [ 0.018529] pinctrl core: initialized pinctrl subsystem [ 0.019274] NET: Registered protocol family 16 [ 0.022011] DMA: preallocated 1024 KiB pool for atomic coherent allocations [ 0.026881] hw-breakpoint: found 5 (+1 reserved) breakpoint and 4 watchpoint registers. [ 0.026888] hw-breakpoint: maximum watchpoint size is 8 bytes. [ 0.027087] Serial: AMBA PL011 UART driver [ 0.028745] bcm2835-mbox 3f00b880.mailbox: mailbox enabled [ 0.029200] uart-pl011 3f201000.serial: could not find pctldev for node /soc/gpio@7e200000/uart0_pins, deferring probe [ 0.060472] bcm2835-dma 3f007000.dma: DMA legacy API manager at bb813000, dmachans=0x1 [ 0.061886] SCSI subsystem initialized [ 0.062110] usbcore: registered new interface driver usbfs [ 0.062162] usbcore: registered new interface driver hub [ 0.062247] usbcore: registered new device driver usb [ 0.070084] raspberrypi-firmware soc:firmware: Attached to firmware from 2018-10-23 14:37 ... (truncated) ... [ 68.131540] usb 1-1.3: new high-speed USB device number 8 using dwc_otg [ 68.262202] usb 1-1.3: New USB device found, idVendor=1a40, idProduct=0101 [ 68.262217] usb 1-1.3: New USB device strings: Mfr=0, Product=1, SerialNumber=0 [ 68.262225] usb 1-1.3: Product: USB2.0 HUB [ 68.265403] hub 1-1.3:1.0: USB hub found [ 68.266375] hub 1-1.3:1.0: 4 ports detected [ 68.581541] usb 1-1.3.2: new full-speed USB device number 9 using dwc_otg [ 68.681553] usb 1-1.3.2: device descriptor read/64, error -32 [ 68.901535] usb 1-1.3.2: device descriptor read/64, error -32 [ 69.121531] usb 1-1.3.2: new full-speed USB device number 10 using dwc_otg [ 69.221528] usb 1-1.3.2: device descriptor read/64, error -32 [ 69.441534] usb 1-1.3.2: device descriptor read/64, error -32 [ 69.561630] usb 1-1.3-port2: attempt power cycle [ 70.221589] usb 1-1.3.2: new full-speed USB device number 11 using dwc_otg [ 70.661591] usb 1-1.3.2: device not accepting address 11, error -32 [ 70.761596] usb 1-1.3.2: new full-speed USB device number 12 using dwc_otg [ 71.201611] usb 1-1.3.2: device not accepting address 12, error -32 [ 71.201718] usb 1-1.3-port2: unable to enumerate USB device ```

Full lsusb (click to expand)

``` Bus 001 Device 008: ID 1a40:0101 Terminus Technology Inc. Hub Couldn't open device, some information will be missing Device Descriptor: bLength 18 bDescriptorType 1 bcdUSB 2.00 bDeviceClass 9 Hub bDeviceSubClass 0 Unused bDeviceProtocol 1 Single TT bMaxPacketSize0 64 idVendor 0x1a40 Terminus Technology Inc. idProduct 0x0101 Hub bcdDevice 1.00 iManufacturer 0 iProduct 1 iSerial 0 bNumConfigurations 1 Configuration Descriptor: bLength 9 bDescriptorType 2 wTotalLength 25 bNumInterfaces 1 bConfigurationValue 1 iConfiguration 0 bmAttributes 0xe0 Self Powered Remote Wakeup MaxPower 100mA Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 0 bAlternateSetting 0 bNumEndpoints 1 bInterfaceClass 9 Hub bInterfaceSubClass 0 Unused bInterfaceProtocol 0 Full speed (or root) hub iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 3 Transfer Type Interrupt Synch Type None Usage Type Data wMaxPacketSize 0x0001 1x 1 bytes bInterval 12 Bus 001 Device 005: ID 0424:7800 Standard Microsystems Corp. Couldn't open device, some information will be missing Device Descriptor: bLength 18 bDescriptorType 1 bcdUSB 2.10 bDeviceClass 255 Vendor Specific Class bDeviceSubClass 0 bDeviceProtocol 255 bMaxPacketSize0 64 idVendor 0x0424 Standard Microsystems Corp. idProduct 0x7800 bcdDevice 3.00 iManufacturer 0 iProduct 0 iSerial 0 bNumConfigurations 1 Configuration Descriptor: bLength 9 bDescriptorType 2 wTotalLength 39 bNumInterfaces 1 bConfigurationValue 1 iConfiguration 0 bmAttributes 0xe0 Self Powered Remote Wakeup MaxPower 2mA Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 0 bAlternateSetting 0 bNumEndpoints 3 bInterfaceClass 255 Vendor Specific Class bInterfaceSubClass 0 bInterfaceProtocol 255 iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 2 Transfer Type Bulk Synch Type None Usage Type Data wMaxPacketSize 0x0200 1x 512 bytes bInterval 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x02 EP 2 OUT bmAttributes 2 Transfer Type Bulk Synch Type None Usage Type Data wMaxPacketSize 0x0200 1x 512 bytes bInterval 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x83 EP 3 IN bmAttributes 3 Transfer Type Interrupt Synch Type None Usage Type Data wMaxPacketSize 0x0010 1x 16 bytes bInterval 4 Bus 001 Device 003: ID 0424:2514 Standard Microsystems Corp. USB 2.0 Hub Couldn't open device, some information will be missing Device Descriptor: bLength 18 bDescriptorType 1 bcdUSB 2.00 bDeviceClass 9 Hub bDeviceSubClass 0 Unused bDeviceProtocol 2 TT per port bMaxPacketSize0 64 idVendor 0x0424 Standard Microsystems Corp. idProduct 0x2514 USB 2.0 Hub bcdDevice b.b3 iManufacturer 0 iProduct 0 iSerial 0 bNumConfigurations 1 Configuration Descriptor: bLength 9 bDescriptorType 2 wTotalLength 41 bNumInterfaces 1 bConfigurationValue 1 iConfiguration 0 bmAttributes 0xe0 Self Powered Remote Wakeup MaxPower 2mA Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 0 bAlternateSetting 0 bNumEndpoints 1 bInterfaceClass 9 Hub bInterfaceSubClass 0 Unused bInterfaceProtocol 1 Single TT iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 3 Transfer Type Interrupt Synch Type None Usage Type Data wMaxPacketSize 0x0001 1x 1 bytes bInterval 12 Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 0 bAlternateSetting 1 bNumEndpoints 1 bInterfaceClass 9 Hub bInterfaceSubClass 0 Unused bInterfaceProtocol 2 TT per port iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 3 Transfer Type Interrupt Synch Type None Usage Type Data wMaxPacketSize 0x0001 1x 1 bytes bInterval 12 Bus 001 Device 002: ID 0424:2514 Standard Microsystems Corp. USB 2.0 Hub Couldn't open device, some information will be missing Device Descriptor: bLength 18 bDescriptorType 1 bcdUSB 2.00 bDeviceClass 9 Hub bDeviceSubClass 0 Unused bDeviceProtocol 2 TT per port bMaxPacketSize0 64 idVendor 0x0424 Standard Microsystems Corp. idProduct 0x2514 USB 2.0 Hub bcdDevice b.b3 iManufacturer 0 iProduct 0 iSerial 0 bNumConfigurations 1 Configuration Descriptor: bLength 9 bDescriptorType 2 wTotalLength 41 bNumInterfaces 1 bConfigurationValue 1 iConfiguration 0 bmAttributes 0xe0 Self Powered Remote Wakeup MaxPower 2mA Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 0 bAlternateSetting 0 bNumEndpoints 1 bInterfaceClass 9 Hub bInterfaceSubClass 0 Unused bInterfaceProtocol 1 Single TT iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 3 Transfer Type Interrupt Synch Type None Usage Type Data wMaxPacketSize 0x0001 1x 1 bytes bInterval 12 Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 0 bAlternateSetting 1 bNumEndpoints 1 bInterfaceClass 9 Hub bInterfaceSubClass 0 Unused bInterfaceProtocol 2 TT per port iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 3 Transfer Type Interrupt Synch Type None Usage Type Data wMaxPacketSize 0x0001 1x 1 bytes bInterval 12 Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Couldn't open device, some information will be missing Device Descriptor: bLength 18 bDescriptorType 1 bcdUSB 2.00 bDeviceClass 9 Hub bDeviceSubClass 0 Unused bDeviceProtocol 1 Single TT bMaxPacketSize0 64 idVendor 0x1d6b Linux Foundation idProduct 0x0002 2.0 root hub bcdDevice 4.14 iManufacturer 3 iProduct 2 iSerial 1 bNumConfigurations 1 Configuration Descriptor: bLength 9 bDescriptorType 2 wTotalLength 25 bNumInterfaces 1 bConfigurationValue 1 iConfiguration 0 bmAttributes 0xe0 Self Powered Remote Wakeup MaxPower 0mA Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 0 bAlternateSetting 0 bNumEndpoints 1 bInterfaceClass 9 Hub bInterfaceSubClass 0 Unused bInterfaceProtocol 0 Full speed (or root) hub iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 3 Transfer Type Interrupt Synch Type None Usage Type Data wMaxPacketSize 0x0004 1x 4 bytes bInterval 12 ```

What speaks against a hardware fault of the dongle:

Please let me know if the hardware could be faulty in a way that makes it still work in the described scenarios. I am wondering if a faulty crystal could cause this.

The dongle is one of the dirt-cheap (3$) generic products available through AliExpress and Ebay. It contains a Terminus Technology SL2.2s USB 2.0 HUB TT0509A18 chip and a CoreChip SR9700 ethernet chip. I have posted pictures of the internals on the Raspberry Pi forums.

Setup:

Used kernel: Newest from rpi-update (4.14.78-v7+) Tested Pis: Pi0, Pi 3B+ Power supplies: Tested 3 different supplies + Powering the dongle itself Connection: Ethernet+Hub dongle connected directly into the Pis

What I have tried:

Please let me know if there's any way I could help fix the issue!

P33M commented 6 years ago

Where can I buy one of these devices? A link to a reputable e-shop selling these as a line item (not an auction for a limited quantity) would be preferred.

FD- commented 6 years ago

I'm afraid I don't know of any reputable shop that sells them (I guess that's why they're so cheap). If you consider AliExpress a reputable shop, I found this product has multiple reviews mentioning the same problems with the Raspberry Pi:

https://www.aliexpress.com/item/Hot-Brand-Micro-USB-to-Network-LAN-Ethernet-RJ45-Adapter-with-3-Port-USB-2-0/32691036545.html

a3desu commented 6 years ago

Have the same problem with BURO BU-HUB4-U2.0-Slim link

It has Terminus Technology SL2.2s USB 2.0 HUB TT0509A18 chip inside

Hi speed devices work ok with it(wifi/bluetooth dongle). Any 1.1 device gives the same dmesg errors(tried ftdi, two keyboards)

Works great with PC.

FD- commented 6 years ago

That's very interesting indeed! The ethernet chip (CoreChip SR9700) is USB 1.1, so the problem actually seems to be with talking to a USB 1.1 device over the Terminus SL2.2s TT0509A18 chip. Maybe this helps to narrow down possible causes, @P33M ?

a3desu commented 6 years ago

@FD-, how did you set up DWC_OTG debug level?

SET_DEBUG_LEVEL in dwc_otg_dbg.h does not called from anywhere. Is it possible to set debug level at runtime?

FD- commented 6 years ago

I also couldn't really make sense of SET_DEBUG_LEVEL, so I chose a somewhat hacky approach: I changed DWC_DEBUGPL to remove the level check:

#define DWC_DEBUGPL(lvl, x...) do{ __DWC_DEBUG(USB_DWC x ); }while(0) 

and changed CHK_DEBUG_LEVEL to always return true:

# define CHK_DEBUG_LEVEL(level) (1)

Unfortunately, this lead to a huge amount of debug messages being logged. I was lucky to be able to extract some of the messages before the system locked up due to no more disk space. Had to reflash the sd card in the end.

I also had to enable debugging in the dwc_common makefile.

Alternatively, I guess you could hard-code the logging level in dwc_otg_driver.c. That should provide a more sane amount of log messages depending on the level you choose. .

a3desu commented 6 years ago

Ok. I thought you find a better way :) I rewrote this macro as:

# define DWC_DEBUGPL(lvl, x...) do{ if ((lvl)&DWC_OTG_DEBUGLEV)__DWC_DEBUG(USB_DWC x ); }while(0)

and use Makefile to set debug level:

ccflags-y += -DDWC_OTG_DEBUGLEV=DBG_HCD

I think you may miss broken pipe moment in your log capture. Because it must be shown up at line like:

... [ 115.087836] _complete: urb b70e0c80, device 6, ep 1 IN, status=0 ...

where status will be -32

FD- commented 6 years ago

Yeah, as mentioned, I could only save a small portion of the log (I think I only have the messages of one second's time). Please post your log if it captures details about the issue!

a3desu commented 6 years ago

Yesterday I was trying to find where EPIPE is comming from. Traced some functions and noticed that i receive STALLED interrupt somewhere nearby.

Now, just captured DBG_HCD messages and yep! Its here

FD- commented 6 years ago

I'm afraid I don't know anything about these interrupts, but the stalling thing is certainly interesting! I wonder if the stalling also happens with a different host driver. Maybe the dwc_otg driver doesn't correctly handle these interrupts? Or are they just indicative of a bug that occurred already before that stall occurs?

a3desu commented 6 years ago

Here is verbose part till status=-32

a3desu commented 6 years ago

And here goes strange thing. DBG_HCDI logs interupts. Each record looks like:

[   59.783905] DWC_otg: DWC OTG HCD Interrupt Detected gintsts&gintmsk=0x02000008 core_if=da425800
[   59.783916] DWC_otg: --Host Channel 5 Interrupt: Channel Halted--
[   59.783924] DWC_otg: --Host Channel 5 Interrupt: Transfer Complete--
[   59.783943] DWC_otg: DWC OTG HCD Finished Servicing Interrupts

Here entry and exit messages and in the middle is a description. They all looks the same from the system start.

After detecting usb1.1 device they look like this

mypiandrew commented 6 years ago

Hi

We found similar problems with Genesys GL850 Hubs

Kernel version 4.9.80 works for us :

rpi-update 5c80565c5c0c7f820258c792a98b56f22db2dd03

4.14.x doesn't appear to work no matter what we try

bumping up to the latest 4.19 kernel however appears to fix the issue... :

BRANCH=next rpi-update

uname -a Linux raspberrypi 4.19.0-v7+ #1157 SMP Tue Oct 23 18:10:04 BST 2018 armv7l GNU/Linux

Would be interested to see if you find the same results.

P33M commented 6 years ago

A "stall" is a genuine USB device condition (i.e. returned status from the hub). What's odd is that going from 4.14 -> 4.19 fixes the issue, as there have been no changes to the dwc_otg driver in that timeframe. The usual suspect for symptoms like these is timing-related breakage provoked by non-standard device implementations - it looks like there is (another) class of hub that doesn't play well with the Pi.

P33M commented 6 years ago

It may be faster if I just get my hands on the device(s) in question: @mypiandrew @FD- if you can spare/donate an example device that exhibits the issue can you email info@raspberrypi.org with the subject "Github issue 2734 - Failing USB hubs for analysis" to get a dialogue started?

FD- commented 6 years ago

It seems @mypiandrew's issue is not the same as the one with the Terminus Technology SL2.2s USB 2.0 HUB TT0509A18 described here. I have now tested kernel versions 4.1, 4.9, 4.14, 4.19 on a Pi0 and a 3B+ and all show the exact same behaviour (error -32 and unable to enumerate). The issue is not fixed in 4.19.

FD- commented 6 years ago

@P33M I'm afraid I just have one unit that I still need for its hub functionality. Also, it seems like shipping would cost about 2x the price of the dongle itself (including shipping from China).

I'd suggest to contact the AliExpress seller I linked above about the possibility of getting a demo unit. I'd think they're willing to help if you prove them your affiliation with Raspberry Pi. After all, it seems many customers buy the dongle for their Pi0s and write bad reviews when they find it incompatible. If you don't have an AliExpress account, I could act as an intermediary, though I'd imagine that would make things a bit complicated, not least because I didn't get my dongle from there.

As mentioned, this product on AliExpress has multiple reviews mentioning the same issue: https://www.aliexpress.com/item/Hot-Brand-Micro-USB-to-Network-LAN-Ethernet-RJ45-Adapter-with-3-Port-USB-2-0/32691036545.html Alternatively, a forum user recently reported the same issue with this AliExpress product: https://www.aliexpress.com/item/Super-Speed-2017-High-Quality-USB-to-RJ45-Ethernet-Network-Adapter-Card-With-3-Ports-USB/32831851624.html

If there are any runtime / compilation flags you want me to try, please let me know!

6by9 commented 6 years ago

I happen to have one of these devices that reports as 1a40:0101 Terminus Technology Inc. Hub and 0fe6:9700 Kontron (Industrial Computer Source / ICS Advent) DM9601 Fast Ethernet Adapter. @P33M You're welcome to borrow it. It's the variant with a micro-USB plug on it, so plugs straight into a Pi0.

The USB hub side has come up immediately for me with a keyboard and mouse plugged in, and the network side seems to be happy too. These are the devices where they have left the MAC address EEPROM off, so they all end up with the same MAC address. They also claim to be USB2.0 when they're actually USB1.1 or something similar. They're known to be a touch ropey!

P33M commented 6 years ago

My suspicion is that the hub chip(s) in question are knock-offs. I can find no reference to a "SL 2.2s" hub on anything that isn't a Raspberry Pi related post. Other Terminus hubs (including the reduced pin count SOIC package FE 1.1 device) have worked successfully when connected to a Pi before. In fact, opening up the model that @6by9 has confirms that it's a FE1.1s inside.

The "genesys GL850" is apparently a real product so I'd be keen to see if this is a special case.

FD- commented 6 years ago

Will try to find out more about that! Would it be possible to post pictures of @6by9's dongle's internals for comparison? I've posted pictures of mine in the forum.

6by9 commented 6 years ago

https://drive.google.com/file/d/17xdr7gKpDpfnMxb73-hEzw4LKDWTSJvN/view?usp=sharing https://drive.google.com/file/d/1fFDP6_KxB_87rLcTJKa2lAScTJLU2JcN/view?usp=sharing

FD- commented 6 years ago

Thanks so much! Am I allowed to post them on the forum? Strangely enough, I could spot a few differences. My dongle doesn't have that black rectangle next to the ethernet socket (C2). Wonder what that is. Also, my dongle doesn't have the cap at C28. Interestingly, the PCB has silk-screened the position and label for these components, but they're unpopulated.

6by9 commented 6 years ago

The abscence of C28 on your board would imply that they really are scrimping on the power. 100uF 16V on my board. P33M also commented that you're missing the magnetics on the ethernet (the black block behind the ethernet socket on mine), so you have no isolation and it may well misbehave. C12, C13, C15, and C18 are apparently 0 ohm links instead. The two crystals seem to be haphazardly mounted.

Thanks so much! Am I allowed to post them on the forum?

If you wish, yes.

FD- commented 6 years ago

Thanks for the explanation and pictures! From the outside, all these adapters looked like they were the exactly the same.

Also, it seems like @P33M is right that the hub chip on mine is likely a knock-off. It doesn't have that Terminus T logo that yours has. Instead, it has some other logo (possibly "CC"; It's so tiny that it's nearly impossible to read.)

May I ask where you got your dongle from and how much it cost?

I'll post these findings on the forum so that people know which of these adapters work with the Pi and which to avoid.

a3desu commented 6 years ago

No way you can avoid them. The only way is to fix broken Pi code. This hub works pretty well on other systems.

6by9 commented 6 years ago

May I ask where you got your dongle from and how much it cost?

Ebay in December 2015. About £5 IIRC - I haven't got the history back that far immediately to hand.

P33M commented 6 years ago

No way you can avoid them. The only way is to fix broken Pi code. This hub works pretty well on other systems.

It remains to be seen whether this can be fixed "in code". We already have documented examples of non-USB2.0 compliant hub chips (particular VIA USB3.0 hubs) misbehaving when connected to the Pi and there is nothing we can do in software to accommodate the hardware incompatibilities. Without plugging a protocol analyser in the loop between the hub and the Pi, the source of the incompatibility is undetermined.

FD- commented 6 years ago

Ebay in December 2015. About £5 IIRC - I haven't got the history back that far immediately to hand.

Thanks! Mine was a bit cheaper (~3 dollars) on ebay, so it seems they tried to make extra savings in the construction.

It remains to be seen whether this can be fixed "in code". We already have documented examples of non-USB2.0 compliant hub chips (particular VIA USB3.0 hubs) misbehaving when connected to the Pi and there is nothing we can do in software to accommodate the hardware incompatibilities. Without plugging a protocol analyser in the loop between the hub and the Pi, the source of the incompatibility is undetermined.

I just found the datasheet for the SL2.2s: https://wenku.baidu.com/view/5bfdec3c182e453610661ed9ad51f01dc28157a9 It's in Chinese and doesn't mention Terminus Tech, which strengthens the theory it's a knockoff. I agree with @ickromwerk that nonetheless, it would be great to have it supported by the Raspberry Pi!

FD- commented 6 years ago

The logo is indeed CC, seemingly for CoreChips. The logo looks identical to that on this chip, though that's not the exact same chip.

FD- commented 6 years ago

I finally got hold of the full datasheet in PDF form, a schematic drawing and a PCB design file: https://archive.org/details/SL2.2sDatasheetV12

I don't know Chinese, so these are of very limited use for me, but maybe someone else can make sense of them? For comparison, here is the datasheet of the Terminus Tech FE1.1s Seems like the manufacturer of the SL2.2s is actually a company called SL Chip ShenZhen Co., Ltd. I don't know why the chip carries a CoreChips logo when it's actually designed by another company. Maybe SL just designed these, but doesn't produce them.

hunnimonstr commented 6 years ago

@pm33, i have 3 x usb hub/network port devices that match photographs of 'failing' versions of these hubs that i bought for use with my Pi-0 dev boards (micro/otg usb plug). @fd over on the Pi user forum pointed me here.. been a while since i plugged in to test but iirc lsusb returned an entry for the additional usb hub and nothing else.. this led me on a late night google scavenge that proved fruitless (probably because its a long time since i reuilt a system kernel)

I am in the UK, So if you want a useless hub to play with let me know your postal addy and i will drop one in the post. the accompanying driver cd proved useless/unreadable on 2 x of the 3 hub-cd bundles i took a punt on.. i guess i can zip and attache the cd contents here?

FD- commented 6 years ago

Just marking @P33M here so he's notified about @hunnimonstr's comment.

P33M commented 6 years ago

@hunnimonstr As per previous comment - I would be interested in getting hold of these.

It may be faster if I just get my hands on the device(s) in question: if you can spare/donate an example device that exhibits the issue can you email info@raspberrypi.org with the subject "Github issue 2734 - Failing USB hubs for analysis" to get a dialogue started?

To confirm, can you post a full dmesg output after the failing hub has been plugged in, so I can confirm that it's the same issue?

P33M commented 5 years ago

@mypiandrew I have responded to your email. info@ remains devoid of other correspondence, though...

P33M commented 5 years ago

I am now in receipt of both the cheap hub and the CM-based hardware. However most of our test equipment is in crates awaiting our office move...

FD- commented 5 years ago

@P33M Just to clarify: By cheap hub, do you mean a hub+ethernet combo or a GL850 Hub?

P33M commented 5 years ago

The issue appears to be linked to full-speed devices behind GL850 hubs or unknown equivalents. I have 2 devices that exhibit the issue which may or may not have identical hub chips.

P33M commented 5 years ago

The failing hub appears to ignore the very first start-split transaction that I send it. It responds with "ACK" but subsequent CSPLIT packets either respond STALL or a sequence of NYET-STALL. Nothing comes out of the other end, i.e. no transactions appear at the device.

Looking at analyzer logs, it looks like we're setting port 1 in the ssplit packet regardless of what the actual port is. That's likely to cause no end of trouble.

P33M commented 5 years ago

If I plug a FS device into the combo hub in port 1, it works. If I plug a FS device into port 1 and another FS device into ports 3 or 4 (port 2 is the internal ethernet adapter which I have disabled) then the device fails to probe in the same fashion. This one-liner fixes the problem on the combo ethernet hub, I will shortly prove the fix works on @mypiandrew's test hardware.

diff --git a/drivers/usb/host/dwc_otg/dwc_otg_hcd_linux.c b/drivers/usb/host/dwc_otg/dwc_otg_hcd_linux.c
index ed96940..99a58ad 100644
--- a/drivers/usb/host/dwc_otg/dwc_otg_hcd_linux.c
+++ b/drivers/usb/host/dwc_otg/dwc_otg_hcd_linux.c
@@ -232,7 +232,7 @@ static int _hub_info(dwc_otg_hcd_t * hcd, void *urb_handle, uint32_t * hub_addr,
                else
                        *hub_addr = urb->dev->tt->hub->devnum;
        }
-       *port_addr = urb->dev->tt->multi ? urb->dev->ttport : 1;
+       *port_addr = urb->dev->ttport;
    } else {
         *hub_addr = 0;
        *port_addr = urb->dev->ttport;

This bug has been latent for ages - years, in fact. It's baffling why there aren't more people periodically tripping over this, because if you ever plug a full or low-speed device into a single-TT hub on anything other than port 1, you're going to have a bad time.

Edit: even weirder is how 4.14 suffers from this issue but 4.19 doesn't. This is obviously incorrect code but the correct value for port_addr would only be returned if tt->multi was always set in 4.19, including for single-TT hubs.

FD- commented 5 years ago

Wow, this is great news! I can confirm the fix works for my HUB+Lan dongle. I wonder how @6by9's dongle was working when this issue likely affected it as well - does it probably have the ethernet adapter attached to port 1?

You did a fantastic job @P33M ! Thanks for your efforts!

FD- commented 5 years ago

@P33M AFAICT, this particular problem (with the HUB+Lan dongle) did occur in 4.19 and 4.14 alike. I'm afraid that likely means @mypiandrew's issue isn't the same then - fingers crossed your fix works for both problems nevertheless!

P33M commented 5 years ago

If a single full-speed device is connected to port 1 then the issue doesn't appear. @mypiandrew's hardware has a full-speed device connected to port 2, as does your adapter - which is what makes the discrepancy between 4.19 (working) and 4.14 (failing) on the myPi board odd.

@mypiandrew can you post dmesg logs in the failing case with 4.14?

P33M commented 5 years ago

Curious. A Terminus Technology hub single TT hub with ID 1a40:0101 will unconditionally broadcast all split transaction packets on ALL active downstream FS/LS ports. It also appears that a Genesys Logic single-TT hub with ID 05e3:0608 hub (as used on the MyPi hardware) behaves the same. The broadcast mechanism means the response from the device is returned to the host as the downstream device responds according to its configured address. Various other sequences and gotchas in the way Linux probes for devices means that there are never two unconfigured (i.e. address 0) devices connected to active ports at the same time.

These two devices comprise the majority of single-TT hubs that we've seen in the field.

However, a "Terminus Technology 1a40:0101" hub with the suspicious marking of SL2.2s on top behaves differently - it requires the port number to match the downstream device.

FD- commented 5 years ago

@P33M I'm afraid I don't know enough about USB to understand the first paragraph, but:

I did research on the SL2.2s and am pretty sure it's not a Terminus Tech product. In fact, I found the datasheet (in Chinese) and it said it was designed by "SL Chip ShenZhen Co.,Ltd". I believe they just chose the name SL2.2s and fake the VID/PID so customers (and existing drivers?) were fooled into believing it was a product of an established brand.

Interestingly, there also are FE1.1s clones that seem to more closely mimic the original. These are labelled "FE1.1 USB 2.0 Hub" (without a "T" Terminus Tech logo). I found one of these in another USB+Ethernet dongle (pictures in the forum thread).

Lastly, it also seems like counterfeit FE1.1s chips exist that carry the original silk screen but are in fact clones (At least that's what this page suggests; I don't know Chinese, but it seems the one framed in red is the fake).

Just to highlight behaviour could differ even for the seemingly same chip.

a3desu commented 5 years ago

@P33L, does it mean that" Terminus Technology 1a40:0101" hub with the suspicious marking of SL2.2s on the chip requires additional driver to work correctly?

IE some defines in kernel config

mypiandrew commented 5 years ago

Hi @P33M Interesting find regarding the one line bug, I did some googling on this and found this related thread which may be of interest.

https://www.spinics.net/lists/linux-usb/msg135414.html

I have found in the past that the “see a problem, fix a problem” fault-finding mantra is especially important with software as all too often a small change/fix in one place has a knock-on effect elsewhere.

Talking around the problem specifically relating to our hardware for a moment, I would be interested to see the result of this change on 4.14 and if it provides a consistent fix across multiple (re)boots. I say this because we found that the error can present intermittently on 4.14, whereas 4.9.80 and 4.19 kernels either side are consistently good regardless of how or how often the unit is (re)booted. To me this implies some kind of race-condition or some small delays added elsewhere are perhaps impacting the issue (I believe we’ve speculated on this before on other threads)

For interest if I look back I can see we ran into a very similar looking USB problems when going from kernel 4.1 to 4.4 (see github issue #1633) and again looking at some internal email history with 4.9.19 which was “fixed” somewhere around 4.9.36.

If you still need them I’ll upload some logs again for a little later today for 4.14, but it will be similar to those I’ve uploaded before.

Also if it would help with productivity let me know if you need another Compute Module shipped over (preloaded with 4.14 OS)

P33M commented 5 years ago

I would suspect that the broadcast behaviour of the hub chips is going to be in "undefined behaviour" territory - what if port 1 is enabled as a high-speed port, and we're trying to talk to port 2 as a full-speed port? Order of operations may be important here - there is no defined sequence in which the kernel will probe USB devices as they are all done in an asynchronous, multi-threaded manner.

I wouldn't be surprised if 4.14 suddenly starts working (across multiple versions) with this fix backported to each one.

mypiandrew commented 5 years ago

what kernel version has this fix in?

mypiandrew commented 5 years ago

Linux raspberrypi 4.14.93-v7+ #1189 SMP Mon Jan 14 16:52:29 GMT 2019 armv7l GNU/Linux

still shows the fault

https://gist.github.com/mypiandrew/184f598ba731ccb2d96f8da5199d33da

popcornmix commented 5 years ago

That kernel is older than the fix. Try updating now - latest rpi-update kernel includes the fix.