search

raspberrypi / linux

Kernel source tree for Raspberry Pi-provided kernel builds. Issues unrelated to the linux kernel should be posted on the community forum at https://forums.raspberrypi.com/
Other
11.11k stars 4.98k forks source link

USB3.0 to SATA adapter causes problems #3070

Closed M-Reimer closed 5 years ago

M-Reimer commented 5 years ago

Describe the bug

I have a SATA SSD connected to a USB3.0 to SATA converter.

If I connect this to my Raspberry Pi 4, then the "Access" LED flashes pretty long and I get errors logged (uas_eh_device_reset_handler start)

To reproduce

This is the adapter: https://www.amazon.de/dp/B079QQJB9L/

If this is used on the Raspberry Pi, it takes pretty long for the kernel to "detect" the SSD drive correctly. I haven't tested reliability while using the drive, so far.

I have tried a powered hub but the errors keep exactly the same.

System

Tried several systems. Up-to-date Raspbian and Arch Linux with several self-compiled kernels. Always the same problem.

Logs

Jul 12 20:38:34 alarmpi kernel: usb 2-1: new SuperSpeed Gen 1 USB device number 4 using xhci_hcd Jul 12 20:38:34 alarmpi kernel: usb 2-1: New USB device found, idVendor=2109, idProduct=8110, bcdDevice=91.04 Jul 12 20:38:34 alarmpi kernel: usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0 Jul 12 20:38:34 alarmpi kernel: usb 2-1: Product: USB3.0 Hub
Jul 12 20:38:34 alarmpi kernel: usb 2-1: Manufacturer: VIA Labs, Inc.
Jul 12 20:38:34 alarmpi kernel: hub 2-1:1.0: USB hub found Jul 12 20:38:34 alarmpi kernel: hub 2-1:1.0: 4 ports detected Jul 12 20:38:34 alarmpi kernel: usb 2-1.1: new SuperSpeed Gen 1 USB device number 5 using xhci_hcd Jul 12 20:38:34 alarmpi kernel: usb 2-1.1: New USB device found, idVendor=152d, idProduct=0578, bcdDevice=32.02 Jul 12 20:38:34 alarmpi kernel: usb 2-1.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3 Jul 12 20:38:34 alarmpi kernel: usb 2-1.1: Product: JMS579 Jul 12 20:38:34 alarmpi kernel: usb 2-1.1: Manufacturer: JMicron Jul 12 20:38:34 alarmpi kernel: usb 2-1.1: SerialNumber: 191951815235 Jul 12 20:38:34 alarmpi kernel: scsi host0: uas Jul 12 20:38:34 alarmpi kernel: scsi 0:0:0:0: Direct-Access CT240BX5 00SSD1 3202 PQ: 0 ANSI: 6 Jul 12 20:38:34 alarmpi kernel: sd 0:0:0:0: [sda] 468862128 512-byte logical blocks: (240 GB/224 GiB) Jul 12 20:38:34 alarmpi kernel: sd 0:0:0:0: [sda] 4096-byte physical blocks Jul 12 20:38:34 alarmpi kernel: sd 0:0:0:0: [sda] Write Protect is off Jul 12 20:38:34 alarmpi kernel: sd 0:0:0:0: [sda] Mode Sense: 53 00 00 08 Jul 12 20:38:34 alarmpi kernel: sd 0:0:0:0: [sda] Disabling FUA Jul 12 20:38:34 alarmpi kernel: sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA Jul 12 20:38:34 alarmpi kernel: sd 0:0:0:0: [sda] Optimal transfer size 33553920 bytes not a multiple of physical block size (4096 bytes) Jul 12 20:38:34 alarmpi kernel: sda: sda1 Jul 12 20:38:34 alarmpi kernel: sd 0:0:0:0: [sda] Attached SCSI disk Jul 12 20:39:05 alarmpi kernel: sd 0:0:0:0: [sda] tag#5 uas_eh_abort_handler 0 uas-tag 6 inflight: CMD IN Jul 12 20:39:05 alarmpi kernel: sd 0:0:0:0: [sda] tag#5 CDB: opcode=0x28 28 00 1b f2 41 d8 00 00 28 00 Jul 12 20:39:05 alarmpi kernel: sd 0:0:0:0: [sda] tag#4 uas_eh_abort_handler 0 uas-tag 5 inflight: CMD IN Jul 12 20:39:05 alarmpi kernel: sd 0:0:0:0: [sda] tag#4 CDB: opcode=0x28 28 00 1b f2 41 28 00 00 a8 00 Jul 12 20:39:05 alarmpi kernel: scsi host0: uas_eh_device_reset_handler start Jul 12 20:39:05 alarmpi kernel: usb 2-1.1: reset SuperSpeed Gen 1 USB device number 5 using xhci_hcd Jul 12 20:39:05 alarmpi kernel: scsi host0: uas_eh_device_reset_handler success Jul 12 20:39:35 alarmpi systemd-udevd[221]: sda: Worker [295] processing SEQNUM=2007 is taking a long time Jul 12 20:39:36 alarmpi kernel: sd 0:0:0:0: [sda] tag#1 uas_eh_abort_handler 0 uas-tag 2 inflight: CMD IN Jul 12 20:39:36 alarmpi kernel: sd 0:0:0:0: [sda] tag#1 CDB: opcode=0x28 28 00 1b f2 44 78 00 00 28 00 Jul 12 20:39:36 alarmpi kernel: sd 0:0:0:0: [sda] tag#0 uas_eh_abort_handler 0 uas-tag 1 inflight: CMD IN Jul 12 20:39:36 alarmpi kernel: sd 0:0:0:0: [sda] tag#0 CDB: opcode=0x28 28 00 1b f2 44 08 00 00 68 00 Jul 12 20:39:36 alarmpi kernel: sd 0:0:0:0: [sda] tag#2 uas_eh_abort_handler 0 uas-tag 3 inflight: CMD IN Jul 12 20:39:36 alarmpi kernel: sd 0:0:0:0: [sda] tag#2 CDB: opcode=0x28 28 00 1b f2 43 b8 00 00 48 00 Jul 12 20:39:36 alarmpi kernel: scsi host0: uas_eh_device_reset_handler start Jul 12 20:39:36 alarmpi kernel: usb 2-1.1: reset SuperSpeed Gen 1 USB device number 5 using xhci_hcd Jul 12 20:39:36 alarmpi kernel: scsi host0: uas_eh_device_reset_handler success

M-Reimer commented 5 years ago

I still have the board with the "network problems": https://github.com/raspberrypi/linux/issues/3034 So I moved the SD card to this board and tried the same SATA adapter. I have exactly the same problems on this second board. I can offer the same as I did in https://github.com/raspberrypi/linux/issues/3034: If it helps I can ship this hardware combination if I get paid for the hardware.

The bug is gone if I add an "usb-storage.quirks=152d:0578:u" entry to cmdline.txt but this actually disables the USB 3.0 mode (slower speed).

I guess getting the new RPi 4 working directly from the start was just too good to be true. There seem to be some driver (or even hardware?) issues left. Maybe one of the downsides of being an early adopter.

M-Reimer commented 5 years ago

I got another USB3 to SATA adapter (different chipset). With this one it works immediately without errors and without additional power supply. Maybe some kind of incompatibility between the JMicron chipset and the VIA USB host?

P33M commented 5 years ago

The quirk setting doesn't disable "USB3.0" mode, it disables UAS. Mass-storage accesses still go at USB3.0 speeds but things like multiple IO threads will be slower.

The adapter VID:PID already has a quirk to disable forced unit addressing but why isn't there a line appearing about a quirks match?

Can you post the output of dmesg after you've plugged the drive in with UAS disabled?

M-Reimer commented 5 years ago

[ 67.911864] usb 2-1: new SuperSpeed Gen 1 USB device number 3 using xhci_hcd [ 67.942905] usb 2-1: New USB device found, idVendor=152d, idProduct=0578, bcdDevice=32.02 [ 67.942921] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3 [ 67.942933] usb 2-1: Product: JMS579 [ 67.942945] usb 2-1: Manufacturer: JMicron [ 67.942957] usb 2-1: SerialNumber: 191951815235 [ 67.945341] usb 2-1: UAS is blacklisted for this device, using usb-storage instead [ 67.945425] usb 2-1: UAS is blacklisted for this device, using usb-storage instead [ 67.945439] usb-storage 2-1:1.0: USB Mass Storage device detected [ 67.946656] usb-storage 2-1:1.0: Quirks match for vid 152d pid 0578: 1800000 [ 67.946782] scsi host0: usb-storage 2-1:1.0 [ 68.952249] scsi 0:0:0:0: Direct-Access CT240BX5 00SSD1 3202 PQ: 0 ANSI: 6 [ 68.953116] sd 0:0:0:0: Attached scsi generic sg0 type 0 [ 68.953948] sd 0:0:0:0: [sda] 468862128 512-byte logical blocks: (240 GB/224 GiB) [ 68.955015] sd 0:0:0:0: [sda] Write Protect is off [ 68.955033] sd 0:0:0:0: [sda] Mode Sense: 47 00 00 08 [ 68.955769] sd 0:0:0:0: [sda] Disabling FUA [ 68.955786] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [ 68.962294] sda: sda1 [ 68.965266] sd 0:0:0:0: [sda] Attached SCSI disk

M-Reimer commented 5 years ago

OK. I think I can close this one. Either a broken USB 3.0 to SATA adapter or bad Linux support at all. I tried the adapter on my Linux desktop system again. It is detected fast and without errors, but so far I never tried to write to the connected SSD. When doing so, I get the error messages I've added below. I I'll send this adapter back to Amazon.

[ 501.549830] usb 2-4: new SuperSpeed Gen 1 USB device number 2 using xhci_hcd [ 501.567512] usb 2-4: New USB device found, idVendor=152d, idProduct=0578, bcdDevice=32.02 [ 501.567517] usb 2-4: New USB device strings: Mfr=1, Product=2, SerialNumber=3 [ 501.567520] usb 2-4: Product: JMS579 [ 501.567523] usb 2-4: Manufacturer: JMicron [ 501.567526] usb 2-4: SerialNumber: 191951815235 [ 501.594683] usbcore: registered new interface driver usb-storage [ 501.599729] scsi host6: uas [ 501.599800] usbcore: registered new interface driver uas [ 501.600185] scsi 6:0:0:0: Direct-Access CT240BX5 00SSD1 3202 PQ: 0 ANSI: 6 [ 501.600827] sd 6:0:0:0: Attached scsi generic sg2 type 0 [ 501.601439] sd 6:0:0:0: [sdb] 468862128 512-byte logical blocks: (240 GB/224 GiB) [ 501.601441] sd 6:0:0:0: [sdb] 4096-byte physical blocks [ 501.601582] sd 6:0:0:0: [sdb] Write Protect is off [ 501.601583] sd 6:0:0:0: [sdb] Mode Sense: 53 00 00 08 [ 501.601846] sd 6:0:0:0: [sdb] Disabling FUA [ 501.601847] sd 6:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [ 501.602086] sd 6:0:0:0: [sdb] Optimal transfer size 33553920 bytes not a multiple of physical block size (4096 bytes) [ 501.604584] sdb: sdb1 [ 501.605841] sd 6:0:0:0: [sdb] Attached SCSI disk [ 521.675076] EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts: (null) [ 573.203294] sd 6:0:0:0: [sdb] tag#29 uas_eh_abort_handler 0 uas-tag 9 inflight: CMD OUT [ 573.203302] sd 6:0:0:0: [sdb] tag#29 CDB: Write(10) 2a 00 00 4f a0 00 00 04 00 00 [ 573.205063] sd 6:0:0:0: [sdb] tag#28 uas_eh_abort_handler 0 uas-tag 10 inflight: CMD OUT [ 573.205070] sd 6:0:0:0: [sdb] tag#28 CDB: Write(10) 2a 00 00 4f a4 00 00 04 00 00 [ 573.208537] sd 6:0:0:0: [sdb] tag#27 uas_eh_abort_handler 0 uas-tag 6 inflight: CMD OUT [ 573.208544] sd 6:0:0:0: [sdb] tag#27 CDB: Write(10) 2a 00 00 4f 9c 00 00 04 00 00 [ 573.212061] sd 6:0:0:0: [sdb] tag#26 uas_eh_abort_handler 0 uas-tag 5 inflight: CMD OUT [ 573.212068] sd 6:0:0:0: [sdb] tag#26 CDB: Write(10) 2a 00 00 4f 98 00 00 04 00 00 [ 573.215562] sd 6:0:0:0: [sdb] tag#25 uas_eh_abort_handler 0 uas-tag 4 inflight: CMD OUT [ 573.215568] sd 6:0:0:0: [sdb] tag#25 CDB: Write(10) 2a 00 00 4f 94 00 00 04 00 00 [ 573.219065] sd 6:0:0:0: [sdb] tag#24 uas_eh_abort_handler 0 uas-tag 3 inflight: CMD OUT [ 573.219072] sd 6:0:0:0: [sdb] tag#24 CDB: Write(10) 2a 00 00 4f 90 00 00 04 00 00 [ 573.222567] sd 6:0:0:0: [sdb] tag#23 uas_eh_abort_handler 0 uas-tag 8 inflight: CMD OUT [ 573.222573] sd 6:0:0:0: [sdb] tag#23 CDB: Write(10) 2a 00 00 4f 84 00 00 04 00 00 [ 573.226065] sd 6:0:0:0: [sdb] tag#22 uas_eh_abort_handler 0 uas-tag 7 inflight: CMD OUT [ 573.226072] sd 6:0:0:0: [sdb] tag#22 CDB: Write(10) 2a 00 00 4f 80 00 00 04 00 00 [ 573.229566] sd 6:0:0:0: [sdb] tag#21 uas_eh_abort_handler 0 uas-tag 2 inflight: CMD OUT [ 573.229572] sd 6:0:0:0: [sdb] tag#21 CDB: Write(10) 2a 00 00 4f 8c 00 00 04 00 00 [ 573.233067] sd 6:0:0:0: [sdb] tag#20 uas_eh_abort_handler 0 uas-tag 1 inflight: CMD OUT [ 573.233074] sd 6:0:0:0: [sdb] tag#20 CDB: Write(10) 2a 00 00 4f 88 00 00 04 00 00 [ 573.236568] sd 6:0:0:0: [sdb] tag#0 uas_eh_abort_handler 0 uas-tag 11 inflight: CMD OUT [ 573.236575] sd 6:0:0:0: [sdb] tag#0 CDB: Write(10) 2a 00 00 4f a8 00 00 04 00 00 [ 573.269992] scsi host6: uas_eh_device_reset_handler start [ 573.393710] usb 2-4: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd [ 573.414256] scsi host6: uas_eh_device_reset_handler success [ 588.128848] audit: type=1130 audit(1563123443.566:45): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=getty@tty5 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' [ 591.551783] audit: type=1006 audit(1563123446.989:46): pid=1673 uid=0 old-auid=4294967295 auid=0 tty=tty5 old-ses=4294967295 ses=6 res=1 [ 605.630016] sd 6:0:0:0: [sdb] tag#29 uas_eh_abort_handler 0 uas-tag 1 inflight: CMD OUT [ 605.630024] sd 6:0:0:0: [sdb] tag#29 CDB: Write(10) 2a 00 00 57 08 00 00 04 00 00 [ 605.630813] sd 6:0:0:0: [sdb] tag#28 uas_eh_abort_handler 0 uas-tag 6 inflight: CMD OUT [ 605.630820] sd 6:0:0:0: [sdb] tag#28 CDB: Write(10) 2a 00 00 56 d4 00 00 04 00 00 [ 605.634285] sd 6:0:0:0: [sdb] tag#25 uas_eh_abort_handler 0 uas-tag 2 inflight: CMD OUT [ 605.634292] sd 6:0:0:0: [sdb] tag#25 CDB: Write(10) 2a 00 00 56 f0 00 00 04 00 00 [ 605.637815] sd 6:0:0:0: [sdb] tag#24 uas_eh_abort_handler 0 uas-tag 3 inflight: CMD OUT [ 605.637821] sd 6:0:0:0: [sdb] tag#24 CDB: Write(10) 2a 00 00 56 e8 00 00 04 00 00 [ 605.641291] sd 6:0:0:0: [sdb] tag#6 uas_eh_abort_handler 0 uas-tag 5 inflight: CMD OUT [ 605.641298] sd 6:0:0:0: [sdb] tag#6 CDB: Write(10) 2a 00 00 56 f4 00 00 04 00 00 [ 605.644810] sd 6:0:0:0: [sdb] tag#5 uas_eh_abort_handler 0 uas-tag 4 inflight: CMD OUT [ 605.644817] sd 6:0:0:0: [sdb] tag#5 CDB: Write(10) 2a 00 00 57 0c 00 00 04 00 00 [ 605.648319] sd 6:0:0:0: [sdb] tag#4 uas_eh_abort_handler 0 uas-tag 11 inflight: CMD OUT [ 605.648325] sd 6:0:0:0: [sdb] tag#4 CDB: Write(10) 2a 00 00 56 ec 00 00 04 00 00 [ 605.651810] sd 6:0:0:0: [sdb] tag#3 uas_eh_abort_handler 0 uas-tag 10 inflight: CMD OUT [ 605.651817] sd 6:0:0:0: [sdb] tag#3 CDB: Write(10) 2a 00 00 56 e4 00 00 04 00 00 [ 605.655309] sd 6:0:0:0: [sdb] tag#2 uas_eh_abort_handler 0 uas-tag 9 inflight: CMD OUT [ 605.655316] sd 6:0:0:0: [sdb] tag#2 CDB: Write(10) 2a 00 00 56 e0 00 00 04 00 00 [ 605.658810] sd 6:0:0:0: [sdb] tag#1 uas_eh_abort_handler 0 uas-tag 8 inflight: CMD OUT [ 605.658816] sd 6:0:0:0: [sdb] tag#1 CDB: Write(10) 2a 00 00 56 dc 00 00 04 00 00 [ 605.662313] sd 6:0:0:0: [sdb] tag#0 uas_eh_abort_handler 0 uas-tag 7 inflight: CMD OUT [ 605.662320] sd 6:0:0:0: [sdb] tag#0 CDB: Write(10) 2a 00 00 56 d8 00 00 04 00 00

scottsweb commented 5 years ago

I have a USB device with the same chipset and I am seeing similar issues:

[ 4877.471127] sd 1:0:0:0: [sdb] Attached SCSI disk
[ 4908.074308] sd 1:0:0:0: [sdb] tag#21 uas_eh_abort_handler 0 uas-tag 6 inflight: CMD IN 
[ 4908.074326] sd 1:0:0:0: [sdb] tag#21 CDB: opcode=0x28 28 00 1b f2 41 d8 00 00 28 00
[ 4908.074463] sd 1:0:0:0: [sdb] tag#20 uas_eh_abort_handler 0 uas-tag 5 inflight: CMD IN 
[ 4908.074477] sd 1:0:0:0: [sdb] tag#20 CDB: opcode=0x28 28 00 1b f2 41 28 00 00 a8 00
[ 4908.114331] scsi host1: uas_eh_device_reset_handler start
[ 4908.265195] usb 2-2: reset SuperSpeed Gen 1 USB device number 4 using xhci_hcd
[ 4908.300379] scsi host1: uas_eh_device_reset_handler success
[ 4938.794714] sd 1:0:0:0: [sdb] tag#25 uas_eh_abort_handler 0 uas-tag 1 inflight: CMD IN 
[ 4938.794731] sd 1:0:0:0: [sdb] tag#25 CDB: opcode=0x28 28 00 00 00 00 00 00 00 08 00
[ 4938.794861] sd 1:0:0:0: [sdb] tag#23 uas_eh_abort_handler 0 uas-tag 3 inflight: CMD IN 
[ 4938.794875] sd 1:0:0:0: [sdb] tag#23 CDB: opcode=0x28 28 00 1b f2 3f 88 00 00 68 00
[ 4938.834710] scsi host1: uas_eh_device_reset_handler start
[ 4938.985557] usb 2-2: reset SuperSpeed Gen 1 USB device number 4 using xhci_hcd

From lsusb:

lsusb
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 002: ID 05dc:a83a Lexar Media, Inc. 
Bus 002 Device 003: ID 152d:0578 JMicron Technology Corp. / JMicron USA Technology Corp. JMS567 SATA 6Gb/s bridge
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Other commands like blkid are locking up the device.

Is it simply a case of this device being not supported correctly?

M-Reimer commented 5 years ago

Is it simply a case of this device being not supported correctly?

I guess so. At least this is not a "Raspberry Issue". I have exactly the same issue on my Linux desktop system (Intel i5). The correct discussion platform would be maybe the kernel mailing list or I think they also have a BugZilla setup somewhere. But I'm too busy to dig deeper into this. Was way easier to order something different and return the problematic adapter to Amazon.

yevmoroz commented 5 years ago

Have you guys checked this thread https://www.raspberrypi.org/forums/viewtopic.php?t=219434 ? My current adapter is following https://www.amazon.de/dp/B00OJ3UJ2S where I have issues just as you describe. I'm thinking of taking this one https://www.amazon.de/gp/product/B018G1RLYC/ as suggested on a thread and try it out.

yevmoroz commented 5 years ago

Also found this thread that provides some explanation to the problem https://www.raspberrypi.org/forums/viewtopic.php?t=245931.

yevmoroz commented 5 years ago

I tried the solution from mentioned thread, just forgot to plug in USB 2.0 port when was testing first. This is actually fine for now until, Thursday when I expect a new adapter :)

yevmoroz commented 5 years ago

Adapter arrived earlier 😏 I can confirm this is working like a charm with a new one. 😎

pelwell commented 5 years ago

What does lsusb show for the new adapter?

yevmoroz commented 5 years ago

Bus 002 Device 002: ID 174c:1153 ASMedia Technology Inc. ASM1153 SATA 3Gb/s bridge

pelwell commented 5 years ago

Thanks - from memory, I also have an adaptor with an ASMedia bridge that works well

yevmoroz commented 5 years ago

I've spent hours everyday trying to understand what is happening. All the time I was thinking it's yet another "Kodi caching issue" (I'm using LibreElec) and was just searching in the wrong direction. Until I remembered that I'm developer and what I can do is just look into log files and this is how I've found this thread and it helped me to solve the root cause of a problem. :smile:

FrostbyteGR commented 4 years ago

This issue shouldn't be closed. I have a few Icy Box IB-183M2 adapters which are based on JMicron's JMS579 chipset and they exhibit the same problem with UAS over USB3.0. The adapters work fine under USB2.0 on the Raspberry Pi 4 and on x86 under USB3.0, without quirks and UAS enabled. This issue probably pertains to VL805 firmware or the Raspberry kernel and both of those are the vendor's responsibility to fix.

JamesH65 commented 4 years ago

This issue probably pertains to VL805 firmware or the Raspberry kernel and both of those are the vendor's responsibility to fix.

We have no access to the source of the VL805 firmware, so we are forced to rely on the manufacturers to fix any problems. Much of the code in the kernel comes from upstream as well, so again, whilst we will do our best to find issues, it's very difficult when it comes from somewhere else and you have no experience in that area.

FrostbyteGR commented 4 years ago

This issue probably pertains to VL805 firmware or the Raspberry kernel and both of those are the vendor's responsibility to fix.

We have no access to the source of the VL805 firmware, so we are forced to rely on the manufacturers to fix any problems. Much of the code in the kernel comes from upstream as well, so again, whilst we will do our best to find issues, it's very difficult when it comes from somewhere else and you have no experience in that area.

I understand your position, but I still believe that we shouldn't be of the mentality "it's not our problem, case closed". Especially so when it's a non-issue on other architectures (x86_64) and on USB2.0 on the affected platform (Raspberry). One of the main reasons behind Raspberry Pi 4's attractiveness is the USB3.0 capabilities (so we're finally spared from the SD card's poor I/O and durability), sadly everything but that works flawlessly. At this point I would also like to point out that JMicron's JMS578 and JMS579 chipsets are very popular choices for the majority of USB-to-SATA/M.2 adapters and that is going to cause a lot of headaches. At the very least, if the team itself is not able to provide with a solution, the issue should be forwarded to whomever is responsible or can provide with a fix. Please keep this issue open and update us in light of new developments.

JamesH65 commented 4 years ago

This issue probably pertains to VL805 firmware or the Raspberry kernel and both of those are the vendor's responsibility to fix.

We have no access to the source of the VL805 firmware, so we are forced to rely on the manufacturers to fix any problems. Much of the code in the kernel comes from upstream as well, so again, whilst we will do our best to find issues, it's very difficult when it comes from somewhere else and you have no experience in that area.

I understand your position, but I still believe that we shouldn't be of the mentality "it's not our problem, case closed". Especially so when it's a non-issue on other architectures (x86_64) and on USB2.0 on the affected platform (Raspberry). One of the main reasons behind Raspberry Pi 4's attractiveness is the USB3.0 capabilities (so we're finally spared from the SD card's poor I/O and durability), sadly everything but that works flawlessly. At this point I would also like to point out that JMicron's JMS578 and JMS579 chipsets are very popular choices for the majority of USB-to-SATA/M.2 adapters and that is going to cause a lot of headaches. At the very least, if the team itself is not able to provide with a solution, the issue should be forwarded to whomever is responsible or can provide with a fix. Please keep this issue open and update us in light of new developments.

"Whilst we do our best to fix issues" does not mean "it's not our problem". We send test cases to VIA when repeatable bugs arise, and we also report and try to fix bug with upstream code when we can. Assuming we just sit on our hands and do nothing is plain rude.

FrostbyteGR commented 4 years ago

This issue probably pertains to VL805 firmware or the Raspberry kernel and both of those are the vendor's responsibility to fix.

We have no access to the source of the VL805 firmware, so we are forced to rely on the manufacturers to fix any problems. Much of the code in the kernel comes from upstream as well, so again, whilst we will do our best to find issues, it's very difficult when it comes from somewhere else and you have no experience in that area.

I understand your position, but I still believe that we shouldn't be of the mentality "it's not our problem, case closed". Especially so when it's a non-issue on other architectures (x86_64) and on USB2.0 on the affected platform (Raspberry). One of the main reasons behind Raspberry Pi 4's attractiveness is the USB3.0 capabilities (so we're finally spared from the SD card's poor I/O and durability), sadly everything but that works flawlessly. At this point I would also like to point out that JMicron's JMS578 and JMS579 chipsets are very popular choices for the majority of USB-to-SATA/M.2 adapters and that is going to cause a lot of headaches. At the very least, if the team itself is not able to provide with a solution, the issue should be forwarded to whomever is responsible or can provide with a fix. Please keep this issue open and update us in light of new developments.

"Whilst we do our best to fix issues" does not mean "it's not our problem". We send test cases to VIA when repeatable bugs arise, and we also report and try to fix bug with upstream code when we can. Assuming we just sit on our hands and do nothing is plain rude.

I was strictly referring to the fact that this specific issue has been closed, the lack of communication regarding this specific issue (and family of adapters) on the corresponding forum thread and the proposed solution being to "simply acquire a compatible adapter" instead. Nowhere did I assume or otherwise state that the team is idling on purpose or not. Let us not derail the discussion any further due to this misunderstanding (and my apologies for that). I propose opening a new issue to track this, since this one has been closed by the author.

FrostbyteGR commented 4 years ago

Can we get an update on the situation or at least a new issue to track this? If there's no plan on fixing JMS579 and JMS578 based adapters, then at least provide us with an official list of known good adapters (both for SATA and M.2 B/B+M key drives) that have been tested and work properly/fully-functional with the Raspberry Pi 4B.

P33M commented 4 years ago

Adapter/device whitelists don't work. There hasn't been, and will never be, a curated set of third-party USB devices that are "known good" because manufacturers frequently change chipset revisions and the firmware that runs on them.

This is why this thread exists (linked above): https://www.raspberrypi.org/forums/viewtopic.php?t=245931

Have you tried connecting your adapters to the USB3.0 port and setting the cmdline.txt quirk?

FrostbyteGR commented 4 years ago

Adapter/device whitelists don't work. There hasn't been, and will never be, a curated set of third-party USB devices that are "known good" because manufacturers frequently change chipset revisions and the firmware that runs on them.

This is why this thread exists (linked above): https://www.raspberrypi.org/forums/viewtopic.php?t=245931

Have you tried connecting your adapters to the USB3.0 port and setting the cmdline.txt quirk?

I am aware of the thread and have perused it extensively. Utilizing quirks (as I mentioned earlier) does work for the time being, but I'd like to have a proper long-term solution (uas instead of usb-storage) in place, to leverage the full capabilities (along with a tiny bit of extra performance and lifespan possibly) that an M.2 SSD can offer. It's just that I'm (among several other users, I suppose) struggling to find such solution. Thank you for understanding and the time/effort you've put into this matter.

SDClass commented 4 years ago

The issue for me stopped when I moved my USB drive from the RPi4's USB 3.0 slot, to the USB 3.0 4-port hub that I added to my system. It's a bit expensive but I like it's size and it's ability to power it from a micro USB cord and 5v power source if needed. So far, I haven't needed to add power to it. It's a bit on the expensive side so I'm assuming any 3.0 hub would also do the trick.

UGREEN USB 3.0 Hub 4 Port Ultra Slim Data Hub with 5V Micro USB Power Compatible for MacBook Mac Pro Mini, iMac, Surface Pro, XPS, IdeaPad, MateBook X Pro, Notebook PC, USB Flash Drives, Mobile HDD

FrostbyteGR commented 4 years ago

Any updates from VIA?

JazzTp commented 4 years ago

[...] so I'm assuming any 3.0 hub would also do the trick. [...]

This one didn't solve the issue, it's possible on the contrary that errors are more frequent than without it, also with a good power supply plugged in:

https://articulo.mercadolibre.com.ar/MLA-857246129-adaptador-hub-usb-30-a-4-puertos-usb-30-alimentacion-exter-_JM?quantity=1#position=37&type=item&tracking_id=ea39649e-9b91-4f50-93f4-bde723f58611

Let me add a rant here, just to put some weight on the "please fix it" side of the thread:

I wonder how many man-hours this thing will have eaten before VIA provides a fix and the fix is made available and well visible for Raspberry PI newcomers, I wish raspberrypi.org had put a very well visible note on their homepage :-|

I'm another of many who's lost some hours (*) discarding other possible sources of the problem, updating upgrading, checking - via another machine - the ssd for bad sectors, getting source code for binaries I was suspecting might have been compiled with libraries not totally compatible with the Raspberry PI and recompiling from source, ordering a powered hub in case the problem was power related (I'm testing the same disk+case on a USB2 port right now... although without that hub because apparently the Raspberry PI won't boot through it on USB2, while it does on USB3... well... I might always go back to booting from an SDcard and putting the external disk in fstab, if not directly as root in /boot/cmdline.txt).

Considering that I'm personally in constant lack of time and considering that this is a deviation from priorities, I have my Dutch friend's words resonating in my ears, "any low end PC will give any Raspberry PI a run for that price".

But I love the small thing idea with the GPIO pins LOL.

FrostbyteGR commented 4 years ago

Hubs are not a valid solution if you have a cluster of Pi's, I am pretty sure the reasons are easy to imagine.

JazzTp commented 4 years ago

Hubs are not a valid solution if you have a cluster of Pi's, I am pretty sure the reasons are easy to imagine.

(I understand that the quoted message, to which I had at first replied, was not addressed to me: I see in my e-mail inbox a post from another user, deleted afterwards apparently, who's running a cluster of 8 Pi's fed via PoE. The comment was actually about hubs not being an acceptable solution.)

FrostbyteGR commented 4 years ago

Hubs are not a valid solution if you have a cluster of Pi's, I am pretty sure the reasons are easy to imagine.

(I understand that the quoted message, to which I had at first replied, was not addressed to me: I see in my e-mail inbox a post from another user, deleted afterwards apparently, who's running a cluster of 8 Pi's fed via PoE. The comment was actually about hubs not being an acceptable solution.)

That user was me, I decided to delete and rewrite my response, because I felt it was maybe too hostile/whiny. Yes, it wasn't addressed to you, I just felt triggered because I was reminded that I was proposed such a solution. I own a PoE-fed cluster of these bad boys and hubs are not a convenient way to go about it, for multiple reasons. I'll just dish out for a new set of adapters and hope their controllers play nice. There go 270+ euros, if the first one proves functional.

JazzTp commented 4 years ago

That user was me, [...]

I see, yes we are a bit annoyed about the issue (and in my case absence of a clear warning on their homepage) and it's easy to become too hostile LOL.

Above, ASMedia Technology Inc. has been mentioned as the way to go for now.

(Problem: most online sellers don't publish what chip their adapters are based on.)

rkkoszewski commented 4 years ago

Just to add some more info to this thread. I have an UpSquared (Not a RaspberryPi but a Intel N4200 CPU single board computer) that has both a VIA USB Host and a JMicron chipset based Hard Drive enclosure and I have the same issue on Proxmox (Debian/Linux) Host. This is definitely not a RaspberryPi issue and has been going on for a year or more for me (No fix in sight, manufacturers when contacted just wash their hands and blame something else as cause).

EDIT: Also, disabling SMART for the drives using the JMicron chipset enclosures reduced dramatically the amount of occurrences of the issue for me (It still rarely happens though).

Treah commented 4 years ago

That user was me, [...]

I see, yes we are a bit annoyed about the issue (and in my case absence of a clear warning on their homepage) and it's easy to become too hostile LOL.

Above, ASMedia Technology Inc. has been mentioned as the way to go for now.

(Problem: most online sellers don't publish what chip their adapters are based on.)

This may solve the problem of data corruption it defiantly does not solve the problem of slow speeds.... I have a adapter with ASMedia and the best I can get on any RPI4 is 30mbs which is USB2.0 speeds with uas enabled. Other systems can transfer data to this drive/adapter at over 200mbs. I suspect this is an issue with the VIA chipset used by the PI4 I was able to find issues with VIA going back to 2008 and having terrible compatibility but not sure if its related. This is very sad for the RPI4 as its one of the biggest selling points for it having a usb3 ports. As it stands alot of adapters to get a ssd working will just suck for speed and makes having usb3 completely pointless. Not sure if the foundation can do anything about this as they stated its an issue with the VIA's stuff but I suspect this probably will never be fixed :(. For now maybe I will try my luck playing the amazon roulette for adapters :/..

M-Reimer commented 4 years ago

I've also seen this issue again recently. My plan was to set up a TV streaming server with recording capability on a RPi 4. To do this I had to add a big storage device. So I ordered an off-the-shelf 4TB USB HDD which, once again, has the issue described here. Recording failed regularly as the HDD was set to read only by the kernel. I disabled UAS for this drive and now everything is rock stable.

This most probably is an issue directly inside the kernel with this JMicron chipset. I wouldn't even say that the VIA chip is involved in this. I had the UAS errors on my Desktop machine, too. Discussing this here may be the wrong place. Someone with at least basic knowledge would have to move this discussion to LKML. If there is no solution for this, then the kernel should at least not even try to use UAS for this chipset.

pelwell commented 4 years ago

The kernel USB subsystem has long been warned about 152d:0578 - see https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v5.4&id=62354454625741f0569c2cbe45b2d192f8fd258e - so it's strange that we still need an explicit quirk.

pelwell commented 4 years ago

What are the USB vendor:device IDs for the JMS578 and JMS579?

FrostbyteGR commented 4 years ago

JMS578 152d:0578 JMS579 152d:1576

pelwell commented 4 years ago

Exactly - now read the linked commit again:

--- a/drivers/usb/storage/unusual_devs.h
+++ b/drivers/usb/storage/unusual_devs.h
@@ -2100,6 +2100,13 @@ UNUSUAL_DEV(  0x152d, 0
...
+UNUSUAL_DEV(0x152d, 0x0578, 0x0000, 0x9999,
+       "JMicron",
+       "JMS567",
+       USB_SC_DEVICE, USB_PR_DEVICE, NULL,
+       US_FL_BROKEN_FUA),
--- a/drivers/usb/storage/unusual_uas.h
+++ b/drivers/usb/storage/unusual_uas.h
@@ -129,6 +129,13 @@ UNUSUAL_DEV(0x152d, 0x0567, 0x0000, 0x9999,
...
+UNUSUAL_DEV(0x152d, 0x0578, 0x0000, 0x9999,
+       "JMicron",
+       "JMS567",
+       USB_SC_DEVICE, USB_PR_DEVICE, NULL,
+       US_FL_BROKEN_FUA),
FrostbyteGR commented 4 years ago

I don't see "1576" anywhere in the commit.

pelwell commented 4 years ago

But you should see 0578.

FrostbyteGR commented 4 years ago

..and how does this help the issue at hand?

Honestly I give up, I'm just gonna go get different chipsets and keep whatever works without issue. I've been nagging about this (JMS579) since March and all we've got is complete radio silence from VIA.

pelwell commented 4 years ago

It was a direct response to @M-Reimer's comment:

If there is no solution for this, then the kernel should at least not even try to use UAS for this chipset.

JazzTp commented 4 years ago

@Treah

This is very sad for the RPI4 as its one of the biggest selling points for it having a usb3 ports. [...]

Agreed (I bought a model which had been out for over an year, PI4B 4GB, to be sure that any such surprises would have been targeted and resolved, as I had read that some raspi hardware in the past had been quickly replaced by a new revision).

Anyway. Let's not rule out already that this can be solved or minimized (meaning not top speed but stable).

I've tried disabling uas. This post explains how (although the author wanted to know how to do it on live systems). This one shows the same method with a different file name (there's just more info to read through to get to the fix).

RESULT

OK booting from SDcard, OK booting from external SSD.

And I could make various tests writing 1-10 GB files to the external SSD reading bytes from /dev/urandom or /dev/zero, not a single crash, obtaining speeds between 6.3 and 114 MBytes/sec (a wide range indeed).

e.g.:

date ; time ( dd if=/dev/zero of=test_upperUSB3_01.dd bs=1M count=1024 ; date ; sync ; date )

However when launching various processes, that is with "real" simultaneous I/O operations, after 2-3 minutes at maximum I was getting a crash + reboot, nothing on the screen, it just went from working fine to black and immediate reboot (with uas active I was getting screens of kernel panic messages and had to switch off and on).

So, in order to collect log info, I went back to booting from the SDcard (UAS also disabled of course), and launched the same processes on the external SSD plugged into USB3.

RESULT

Surprise, no crash so far, it has been running for 45 minutes already.

If/when it crashes, I'll reboot and look for info in log files.

bjoernricks commented 4 years ago

Hi,

I have found this issue while having connection problems with the 2.4GHz Wifi on my new Raspi 4b. It worked without problems at the beginning but suddenly it couldn't connect to the Wifi network anymore. After playing around for several hours I found out that the 5 GHz network works and thought that something must be wrong with the Wifi firmware or the regulation settings. But I couldn't get the 2.4 GHz network to a working state.

Some days later I suddenly remembered the thing I've changed between working and not working 2.4 GHz Wifi. I had plugged in an external USB 3.0 hard disk which I didn't used yet because of the network problems. After removing the device the Wifi just worked as expected again. My USB device uses a JMS579 (152d:1576) and if that one is plugged in during boot the Wifi network just doesn't work. It is reproducible with every boot. Sadly deactivating uas has no influence on this behavior. The only solution for me was to connect the hard disk to a USB 2.0 slot. I really don't know why this has influence to the Wifi driver and I can't imagine what's happening here.

max17048 commented 4 years ago

I have found this issue while having connection problems with the 2.4GHz Wifi on my new Raspi 4b. It worked without problems at the beginning but suddenly it couldn't connect to the Wifi network anymore. After playing around for several hours I found out that the 5 GHz network works and thought that something must be wrong with the Wifi firmware or the regulation settings. But I couldn't get the 2.4 GHz network to a working state.

2.4GHz: Try to disable Bluetooth in /boot/config.txt. Check #3727

bjoernricks commented 4 years ago

2.4GHz: Try to disable Bluetooth in /boot/config.txt. Check #3727

Both work with using USB 2.0 for my hard disk.

pelwell commented 4 years ago

USB3 is known to interfere with the 2.4G WiFi band - this is a general problem. Search for "usb3 wifi interference" for more information.

bjoernricks commented 4 years ago

USB3 is known to interfere with the 2.4G WiFi band - this is a general problem. Search for "usb3 wifi interference" for more information.

Thanks. I wasn't aware of that.

rkkoszewski commented 4 years ago

I've also seen this issue again recently. My plan was to set up a TV streaming server with recording capability on a RPi 4. To do this I had to add a big storage device. So I ordered an off-the-shelf 4TB USB HDD which, once again, has the issue described here. Recording failed regularly as the HDD was set to read only by the kernel. I disabled UAS for this drive and now everything is rock stable.

This most probably is an issue directly inside the kernel with this JMicron chipset. I wouldn't even say that the VIA chip is involved in this. I had the UAS errors on my Desktop machine, too. Discussing this here may be the wrong place. Someone with at least basic knowledge would have to move this discussion to LKML. If there is no solution for this, then the kernel should at least not even try to use UAS for this chipset.

Disabling UAS does not fix the issue, but merely makes it harder to hit the condition for it to fail. At least on my JMS561 (Dual hard drive bay) when both drives are reading or writing on full speed it manages to hit the issue even with UAS disabled for the drives (It takes longer though). Perhaps on single drive enclosures disabling UAS could mitigate the issue better.

skeller commented 4 years ago

I want to add that disabling UAS for JMS578 is not the solution. I have several JMS578 enclosures and they all work fine on different laptops and desktop machines (all Intel based). In UAS mode they outperform by far classic mass-storage. Also they support SCSI UNMAP -> SATA TRIM mapping which also works fine (and allows you to use fstrim e.g.). Given that the Via hub in the Pi4 has problems with other hardware as well (e.g. I have a full-speed battery charger which isn't even recognized at all unless I put a 2.0 hub inbetween), I'm quite sure that the problem is the crappy Via USB implementation and not the JMS bridge. @pelwell Why do you think the JMS is "strange"? Just because of that FUA quirk? That is really a non-issue.

pelwell commented 4 years ago

I don't think the JMS is strange - the "it" in that sentence does not refer to the JMS.