Closed wrobelda closed 10 years ago
The stack trace dump appears corrupted after posting, here's the source for it: http://dawidwrobel.com/files/usb_storage_oops.txt
Seems to be a null pointer defererence in dwc_otg/usb. @ghollingworth ? @P33M?
kernel 3.8.4 compile
CC [M] drivers/usb/host/dwc_otg/dwc_otg_pcd_intr.o drivers/usb/host/dwc_otg/dwc_otg_pcd_intr.c: In function 'dwc_otg_pcd_handle_out_ep_intr': drivers/usb/host/dwc_otg/dwc_otg_pcd_intr.c:4279:21: warning: operation on 'status.d32' may be undefined [-Wsequence-point]
I've seen several OOPSes during driver shutdown, port disconnect, device reset and the like. Sporadic and hard to find.
I note that you are using quite wide frequency ranges and an overclock at 850MHz - does this happen with a fixed 700MHz default frequency?
I ´m I am use 850 MhZ on my type B Pi´s .
But i don´t have any ops with kernel 3.6.11 and up.
But It happend at 3.2.27 i.e on opensuse , fc remix 17.x
Thats why i am here :-) . I thank for popcornmix and others work past weeks to get the dwc_otg fixed with SDHC/SDHX .
Note : I had Ops when using external HDD without an Powered HUB.
@P33M it happens also when not overclocked, but I didn't test with a fixed 700 MHz. I will and let you know.
Running plain 700 MHz and got the following in dmesg. On a first look it seems unrelated (br), but then again eth runs off the USB, so I thought you might want to have a look:
[33612.297448] usb 1-1.3.6: reset high-speed USB device number 6 using dwc_otg
[33850.217140] ------------[ cut here ]------------
[33850.217330] WARNING: at net/bridge/br_multicast.c:284 br_multicast_port_group_expired+0x74/0x78 [bridge]()
[33850.217358] Modules linked in: xt_TCPMSS xt_tcpmss xt_tcpudp iptable_mangle ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 ip_tables x_tables aes_generic bridge ipv6 stp llc pppoe pppox ppp_generic slhc snd_bcm2835 snd_usb_audio snd_pcm snd_page_alloc snd_hwdep snd_usbmidi_lib snd_seq_midi snd_seq_midi_event snd_rawmidi arc4 snd_seq snd_timer snd_seq_device rt2800usb rt2800lib crc_ccitt rt2x00usb rt2x00lib mac80211 snd evdev cfg80211 rfkill leds_gpio led_class
[33850.217600]
sorry, corrupted again: http://www.dawidwrobel.com/files/br_oops.txt
OK, had another USB storage kernel oops this morning (same as initially reported) with fixed 700 MHz frequency, so frequency scaling is not the cause here.
Funny thing is that it always happens in the morning. The only pattern that comes to mind and makes sense with this scenario is that after not being used for a long time at night, some memory swapping occurs (?) and whenever I connect to WiFi in the morning, it oops exactly at this very moment. Is there an equivalent to "dmesg -T" in kdb that would help me to confirm the timing of the oops?
did you disable any sort of power management? say USB? or WiFi?
@wrobelda may you use an POWERD USB HUB ?
Model B require ~900 mA for itsself under load at least for Stable work.
With external HDD the power exhaused with standard ( 1000 mA ) power sources.
@licaon-kter I did not. EDIT: However the HDD itself is by default set to go into suspend mode (spin down) when not used for a while. It must be reported to USB subsystem and the "reset high-speed USB device number 6 using dwc_otg" messages printed upon disk wake-up seem to confirm that. So essentially some power saving is in place for the HDD, but I did not enable it explicitly and it is configured to work this way on its own. I bet that by changing the power saving parameters with hdparm I would make the symptoms go away, but not really solve the underlying cause. Besides that, I do want to have the HDD to spin down when not used.
@remsnet "Plugable 7 Port" hub that I use is powered and said to be one of the most compatible with Raspberry. The HDD is 2,5" inch USB-powered with no external power source connector, so I seriously doubt it would exhaust the standard USB limits.
BTW, something interesting I sometimes also notice:
[pon kwi 15 22:04:54 2013] sd 0:0:0:0: [sda] Device not ready
[pon kwi 15 22:04:54 2013] sd 0:0:0:0: [sda]
[pon kwi 15 22:04:54 2013] Result: hostbyte=0x00 driverbyte=0x08
[pon kwi 15 22:04:54 2013] sd 0:0:0:0: [sda]
[pon kwi 15 22:04:54 2013] Sense Key : 0x2 [current]
[pon kwi 15 22:04:54 2013] Info fld=0x0
[pon kwi 15 22:04:54 2013] sd 0:0:0:0: [sda]
[pon kwi 15 22:04:54 2013] ASC=0x4 ASCQ=0x2
[pon kwi 15 22:04:54 2013] sd 0:0:0:0: [sda] CDB:
[pon kwi 15 22:04:54 2013] cdb[0]=0x28: 28 00 00 74 00 4f 00 00 08 00
[pon kwi 15 22:04:54 2013] end_request: I/O error, dev sda, sector 7602255
@wrobelda , problem looks like the USB POWER EXhaused !! issue seems not not the compatible Standard Hub, as Linux SEE the device correctly.
What it looks for me been that kernel starts to load , and at the point where the kernel access the drive UNDER LOAD ( kernel boot IO load..) the USB Power seem exhaused. This shuold never ever happen . Your 2,5 HDD seems to exhause what your RPI´s powersource can deliver via the RPI USB Conector.
Same will Issue happen on some old Laptops that don´t deliver enoth Power.
I HAD the same with 64G & 128G USB sticks on RPI 3.x kernal equal withóut extra Power for the PI
I use an 7 port belkim POWERD with 2500 mA , have 7 x 64G sticks attached witz raid5 fuse-zfs Such Hub costs here in the EU arround 20 eur ( www.conrad.com i.e )
As written : 900mA for the PI and addional arround 1500mA für exernal disk see your HDD disk specs. My PI´s has an own USB Power Supply with 1200mA witch fully used under PI max load.
"Your 2,5 HDD seems to exhause what your RPI´s powersource can deliver via the RPI USB Conector." @remsnet As I already explained, my HDD is connected to the HUB - just like the wifi card. Only the soundcard connects directly to raspberry's second USB port. I can reconnect it to the HUB but seriously doubt it will make any difference.
reconect the external disk and SOUND Card to the HUB and try again. http://www.seagate.com/internal-hard-drives/enterprise-hard-drives/hdd/constellation/ 3.9 watts (idle) , 20 watt unter load.
http://www.ti.com/lit/ds/symlink/pcm2900c.pdf saying +400mA for the USB Sound if i not read that false.
Its KNOWN that the >>Pis Connector << can´t deliver that amount.
@remsnet the external disk IS connected to the HUB.
Just to asume required Power under Full Load of your RPI Envirement look like
@remsnet: I believe your numbers may be a little exaggerated. For example I measured my RaspberryPi model B under load and it never reaches 500mA.
@wrobelda: Are you sure your HDD can be run from one USB 2.0 port? Maybe it's designed for using it with Y shaped cables? Or for USB 3.0 where you have 900mA per port? Even then, however, it should be able to run on 1000mA max. It does seems to me that the HDD may be a problem (there must be a reason for those I/O errors). It may go to powersafe mode at night and then try to spin in the morning (and take much more power at that time). This may cause some voltage drop. Maybe you should consider powering RaspberryPi from separate power supply just to see if there is any difference?
@kadamski it's 2.0 and designed to be used with one port only. I can try to power Raspberry with external USB power source, but even if this worked it still wouldn't explain why did it start to fail all of sudden after upgrading the firmware, whereas previously I could get it running without problems even when overclocked quite noticeably? I will also try to downgrade the firmware/kernel pair sometime this week to see whether it is back to working state again.
Also, on a related note, it seems that one of these crashes also corrupted one of HDD's partition (XFS filesystem) to the extent of causing an immediate kernel oops upon attempting to mount it on both RaspberyPI and my Ubuntu 13.04 laptop. Just mind-blowing.
So your RPi shouldn't use more than 500mA, your HDD shouldn't use more than 500mA and you have 2500mA power adapter so I believe you should be safe. But of course you can never trust that those cheap power adapters work as they should. Downgrading firmware to find last version working reliably may be the best option to find the reason of the problem, then.
@wrobelda "[pon kwi 15 22:04:54 2013] end_request: I/O error, dev sda, sector 7602255" I suppose you should check the drive SMART status (ie. with smartctl) and maybe run a {short,long} test on it. Your issues could be connected with drive failure.
About hard drives - if disk comes with single link USB cable it should work fine with it. Disk uses more power during spin up, so it could be connected with your comments (quote "that it always happens in the morning").
Kernel 3.8.7 kernel build error
LD drivers/usb/built-in.o ld: error: drivers/usb/host/built-in.o uses VFP register arguments, drivers/usb/built-in.o does not ld: failed to merge target specific data of file drivers/usb/host/built-in.o make[2]: * [drivers/usb/built-in.o] Fehler 1 make[1]: * [drivers/usb] Fehler 2 make: *\ [drivers] Fehler 2
^C[1]+ Exit 2 nohup make CPPFLAGS="-Ofast -mfpu=vfp -mfloat-abi=hard -march=armv6zk -mtune=arm1176jzf-s" CFLAGS="-Ofast -mfpu=vfp -mfloat-abi=hard -march=armv6zk -mtune=arm1176jzf-s" dep zImage Modules
Tried to force hard-float.
anyone hints ?
The kernel does not use floating point anyway, afaik, so forcing has no effect.
@licaon-kter okay , issue resolved then , closed MY issue see https://github.com/raspberrypi/linux/issues/276
Issue resolved - was due to a faulty USB drive. Everything is stable now.
This issue is still on unfortunately. I assumed it was gone because the system was stable after I replaced HDD with pendrive temporarily, so I could RMA the HDD. I received a new disk couple of days ago and have experienced the same issue every day since. Here are some kdb dumps I managed to grab: http://www.dawidwrobel.com/files/rt2x00.txt http://www.dawidwrobel.com/files/kpanic.txt http://www.dawidwrobel.com/files/110.txt http://www.dawidwrobel.com/files/kpanic2.txt
I just looked into dmesg and noticed a lot of the following again:
[ 1302.346947] sd 0:0:0:0: [sda] Unhandled error code [ 1302.347003] sd 0:0:0:0: [sda] [ 1302.347024] Result: hostbyte=0x00 driverbyte=0x00 [ 1302.347043] sd 0:0:0:0: [sda] CDB: [ 1302.347059] cdb[0]=0x28: 28 00 00 04 60 48 00 00 08 00 [ 1302.347115] end_request: I/O error, dev sda, sector 286792 [ 1302.347972] XFS (sda): metadata I/O error: block 0x46048 ("xfs_trans_read_buf_map") error 5 numblks 8 [ 1702.313900] usb 1-1.3.6: reset high-speed USB device number 5 using dwc_otg
Let me repeat, that the USB HDD is attached to the HUB. The Pi is not overclocked - in fact, no configuration option is set at all except for GPU memory adjusted to 128 MB. No USB device is attached to Pi, except for the HUB itself.
To be honest, I lost my faith in having this configuration stable EVER. I had previously run Allwinner A10 based Mele A1000 device for almost a year without any of these issues, and it only had a community support for its kernel and firmware, to which I happily contributed. I am not a fan of RPi anymore - it's hard to stay positive about the concept when the basics fail to work. My USB serial console cable literally got physically broken today because of having constantly connecting it on and off to grab the kdb dump. I just wanted to have a nice, stable, low-power HTPC with a low power 1TB drive attached. It's hard to justify the amount of time I spent so far to have this **\ working.
It could be a problem with USB dequeing since it seems to make the Ethernet drop out.
That would mean there must be something opening and closing one of the USB devices.
What are you doing whilst this is happening? Is it sat doing nothing or are you running some kind of script
Gordon
On 10 May 2013, at 21:58, "Dawid Wr?bel" notifications@github.com<mailto:notifications@github.com> wrote:
This issue is still on unfortunately. I assumed it was gone because the system was stable after I replaced HDD with pendrive temporarily, so I could RMA the HDD. I received a new disk couple of days ago and have experienced the same issue every day since. Here are some kdb dumps I managed to grab: http://www.dawidwrobel.com/files/rt2x00.txt http://www.dawidwrobel.com/files/kpanic.txt http://www.dawidwrobel.com/files/110.txt http://www.dawidwrobel.com/files/kpanic2.txt
I just looked into dmesg and noticed a lot of the following again:
[ 1302.346947] sd 0:0:0:0: [sda] Unhandled error code [ 1302.347003] sd 0:0:0:0: [sda]
[ 1302.347024] Result: hostbyte=0x00 driverbyte=0x00 [ 1302.347043] sd 0:0:0:0: [sda] CDB: [ 1302.347059] cdb[0]=0x28: 28 00 00 04 60 48 00 00 08 00 [ 1302.347115] end_request: I/O error, dev sda, sector 286792 [ 1302.347972] XFS (sda): metadata I/O error: block 0x46048 ("xfs_trans_read_buf_map") error 5 numblks 8 [ 1702.313900] usb 1-1.3.6: reset high-speed USB device number 5 using dwc_otg
To be honest, I lost my faith in having this configuration stable EVER. I had previously run Allwinner A10 based Mele A1000 device for almost a year without any of these issues, and it only had a community support for its kernel and firmware.
Reply to this email directly or view it on GitHubhttps://github.com/raspberrypi/linux/issues/268#issuecomment-17744192.
@wrobelda Are you sure your new drive is good? This I/O error looks bad…
The enclosure was new, but it could still be a refurbished hdd inside. I will test it and get back to you.
I tested the replaced disk some time ago and it turned out to be OK.
Something interesting happened today, though. After turning my PC on in the morning, I could not access the hdd connected to Raspberry. dmesg shown the following: http://www.dawidwrobel.com/files/usb_storage_oops_3.txt
This is a bit different from the previous reports in a way that it does not report any I/O issues, just the usb-storage process hanging. So I disconnected the drive from USB port and connected to my laptop's - it was detected just fine, there were no FS errors in dmesg whatsoever. Interestingly, raspberry did not hang and only showed "usb 1-1.3.5: USB disconnect, device number 5" in dmesg. After connecting again, there was no dmesg activity at all this time. I tried multiple times. The disk, however, was still reported correctly in lsusb: "Bus 001 Device 005: ID 0bc2:2300 Seagate RSS LLC Expansion Portable".
After attempting to mount the FS, serial console hung, but I still had working WiFi connection, so Raspberry did not hang. Therefore it is quite clear the usb-subsystem is suffering from USB issues.
There's too much going on here to begin to determine what the root cause is.
Yes there are still issues with the USB driver. It's just a question of whether you are seeing something new or something I already know about and am working on. You should note that crashes related to dwc_otg_hcd_urb_dequeue and friends are known about and are on my to-do list.
The thread of the issue has evolved somewhat since you first reported kernel OOPSes. Your kernel version appears to also have changed. There have also been several commits to USB since you first reported the problem.
What is the minimum set of circumstances to replicate this broken behaviour on your USB HDD? I assume that is the one device that is the common thread throughout.
Please post lsusb -v for the devices in question.
Please try to replicate with
my issue are Gone with 3.11.6 , close it if you wisch
Hi.
After upgrading the firmware and kernel packages to 20130405, I experience random kernel oops when my USB HDD wakes up from idle.
The hardware is RaspberryPi type B, 512MB, being powered by Plugable 7 Port High Speed USB 2.0 Hub - commonly recommended for compatibility with raspberry. The HUB is connected to Seagate 2,5" HDD and a RTL3072 wifi card that is operating in Master mode (AP) using hostapd. Additionally, an USB sound card is connected to raspberry's second USB port.
Software wise, the system runs raspbian off the HDD. SD is only being used for /boot partition.