search

raspberrypi / linux

Kernel source tree for Raspberry Pi-provided kernel builds. Issues unrelated to the linux kernel should be posted on the community forum at https://forums.raspberrypi.com/
Other
11.03k stars 4.95k forks source link

DVB-C USB transport stream broken #82

Closed r10r closed 10 years ago

r10r commented 12 years ago

Hi,

I need some help debugging a USB transfer problem on my raspberry pi.

The problem is that the DVB Transport stream recorded from a USB DVB stick (cinergy htc stick) attached to a usb hub which is attached to the raspberry pi is broken because packages are missing. I plugged the USB DVB stick directly into the Pi's USB Port (with additional power) with the same results. I use the latest media_build from linuxtv.org against the 3.1.9 raspberry pi kernel. The same drivers from the media_build work perfectly with a 3.1.9 kernel on my thinkpad.

When turning on debugging in the DVB driver I get the following messages:

[  126.162705] tda18271: performing RF tracking filter calibration
[  128.056572] tda18271: RF tracking filter calibration complete
[  128.057855] em28xx #0/2-dvb: Using 5 buffers each with 64 x 940 bytes
[  129.052712] em28xx #0/2-dvb: ISO ERROR COUNT: 1.
[  129.052743] em28xx #0/2-dvb: URB packet 0, status -4001 [Unknown].
[  129.581352] em28xx #0/2-dvb: ISO ERROR COUNT: 1.
[  129.581384] em28xx #0/2-dvb: URB packet 0, status -4001 [Unknown].
[  129.872090] em28xx #0/2-dvb: ISO ERROR COUNT: 1.
[  129.872122] em28xx #0/2-dvb: URB packet 0, status -4001 [Unknown].
[  129.969224] em28xx #0/2-dvb: ISO ERROR COUNT: 1.
[  129.969258] em28xx #0/2-dvb: URB packet 44, status -4001 [Unknown].

My assumption is that the isochronous transfer is broken for the usb host driver.

The error code is defined in:

 drivers/usb/host/dwc_common_port/dwc_os.h:#define DWC_E_NO_STREAM_RES   4001

which is caused by a _DWC_OTG_HC_XFER_FRAMEOVERRUN in the URB:

 dwc_otg/dwc_otg_hcd_intr.c:            frame_desc->status = -DWC_E_NO_STREAM_RES;

I'm a bit lost since I do not have any USB specific knowledge but I really want to get the DVB Stick running. What can I do for further testing/debugging? Thanks for your help!

Cheers, Ruben

Dexterp37 commented 12 years ago

I guess this is closely related to the problem we are experiencing with the Kinect as documented here

r10r commented 12 years ago

@Dexterp37 Do you have any solution for this problem? Or is there any work in progress?

Dexterp37 commented 12 years ago

Sadly.. no :( I noticed that latest kernel source has update USB driver so I'm trying to compile the kernel from source and see if it helps.

r10r commented 12 years ago

Which kernel are you trying? The latest https://github.com/raspberrypi/linux.git from @popcornmix ? Please let me know if it works for you.

Dexterp37 commented 12 years ago

At the moment I'm trying to compile the one from @popcornmix . I'll also give 3.2.x from https://github.com/bootc/linux/tree/rpi-3.2.23 a try. I'm still fighting with some cross compilation/fork issues with Cygwin on Windows.

r10r commented 12 years ago

I recommend you to use a debian VM for cross compilation. Setting up the toolchain there is really simple. Last version I tried was 921a3d1180d23cfc8a75eb2a0288b3b2fb9f1fdb - but maybe the newest commit df9a56229363ae095a7687df117f3f79ba42a20d fixes something.

r10r commented 12 years ago

@Dexterp37 The error code has changed, but unfortunately USB isoc transfer is still broken with the latest raspberry pi kernel:

[   71.135721] drxk: frontend initialized.
[   71.195037] tda18271 0-0060: creating new instance
[   71.199269] TDA18271HD/C2 detected @ 0-0060
[   71.262233] drxk: status = 0x639260d9
[   71.262270] drxk: detected a drx-3926k, spin A3, xtal 20.250 MHz
[   75.581431] DRXK driver version 0.9.4300
[   75.927370] DVB: registering new adapter (em28xx #0)
[   75.927412] DVB: registering adapter 0 frontend 0 (DRXK DVB-C DVB-T)...
[   75.938859] em28xx #0: Successfully loaded em28xx-dvb
[   75.938895] Em28xx: Initialized (Em28xx dvb Extension) extension
[  122.687711] tda18271: performing RF tracking filter calibration
[  124.590611] tda18271: RF tracking filter calibration complete
[  124.591786] em28xx #0/2-dvb: Using 5 buffers each with 64 x 940 bytes
[  126.187359] em28xx #0/2-dvb: ISO ERROR COUNT: 1.
[  126.187393] em28xx #0/2-dvb: URB packet 11, status -63 [Buffer error (overrun)].
[  126.195557] em28xx #0/2-dvb: ISO ERROR COUNT: 1.
[  126.195581] em28xx #0/2-dvb: URB packet 0, status -63 [Buffer error (overrun)].
r10r commented 12 years ago

@Dexterp37 You can try the patch from https://github.com/raspberrypi/linux/issues/29#issuecomment-6884170. With this patch I can at least use the TPLink wireless dongle and my USB keyboard together.

popcornmix commented 12 years ago

Can you try latest firmware with: dwc_otg.microframe_schedule=1 added to cmdline.txt

r10r commented 12 years ago

Hi thanks for the info. I already tried 0872b20fbea2377c6286c39cbc60d3bd83a814aa with the microframe scheduler patch? I already tried the microframe scheduler patch before - USB is more stable, but isoc transfer is still broken.

Dexterp37 commented 12 years ago

I can confirm that the transfer is still broken. Tried this morning with latest kernel + microframe schedule and Microsoft Kinect.

ghollingworth commented 12 years ago

Can you do a lsusb -v to output the endpoint information, just be interested to see if it's highbandwidth isochronous endpoints that are the problem...

Likely issue is that it's just the same high latency problem we're seeing elsewhere (i.e. interrupts are disabled for long enough that we miss processing the isochronous requests until the source overrun's its buffers)

Gordon

Dexterp37 commented 12 years ago

@ghollingworth this is my lsusb -v for Microsoft Kinect. This was done by directly connecting to the pi, without an hub. If you need I can provide a lsusb -v listing with an hub too.

http://pastebin.com/wxY7NQCT

r10r commented 12 years ago

@ghollingworth you can find the output from lsusb -v at https://gist.github.com/3549890. The output from the DVB-C stick is below the full output.

Are you working on this issue Gordon? Is this similar to the spinlock problem in the sdhci driver?

r10r commented 12 years ago

direct link is https://gist.github.com/3549890#file_terratec%20dvb_c%20htc%20stick

ghollingworth commented 12 years ago

Ah...

It seems that they are indeed using high bandwidth transfers... I can believe that they're not going to work with the current driver...

I do have some ideas about how I can get this working one of which is to push off the low latency work onto the multimedia processor (the thing most people think is the GPU!) the other is to implement the low latency work in the FIQ rather than in the IRQ.

Am going to have to first fix the split transaction issues first Then I can have a look at this

Thanks

Gordon

On 31 August 2012 08:43, Ruben Jenster notifications@github.com wrote:

direct link is https://gist.github.com/3549890#file_terratec%20dvb_c%20htc%20stick

— Reply to this email directly or view it on GitHubhttps://github.com/raspberrypi/linux/issues/82#issuecomment-8184770.

r10r commented 12 years ago

@ghollingworth Hey Gordon - sounds good. Just drop me a message if you need someone for testing.

Good luck! Ruben

rmoszczynsk commented 11 years ago

Hi,

I was just wondering if anyone made any progress on this. I'm experiencing the same issue (USB buffer overruns) with a PCTv Nanostick T2 TV dongle. This results in a lot of lost MPEG packets when trying to record a TV stream.

licaon-kter commented 11 years ago

Latest kernel & firmware? If so, neither does smsc95xx.turbo_mode=N nor dwc_otg.speed=1 help separately or together in cmdline.txt ?

ghollingworth commented 11 years ago

Currently I don't have a device like that so cannot comment on any progress.

Is it a USB2.0 device?

Thanks

Gordon

On 27/09/2012, grizlak notifications@github.com wrote:

Hi,

I was just wondering if anyone made any progress on this. I'm experiencing the same issue (USB buffer overruns) with a PCTv Nanostick T2 TV dongle. This results in a lot of lost MPEG packets when trying to record a TV stream.

Reply to this email directly or view it on GitHub: https://github.com/raspberrypi/linux/issues/82#issuecomment-8928779

Sent from my mobile device

r10r commented 11 years ago

If possible I'll try out the current kernel this evening with the given options and give you feeback.

Regards, Ruben

r10r commented 11 years ago

@ghollingworth The terratec cinergy htc stick is a USB 2.0 device

rmoszczynsk commented 11 years ago

The TV device I have is USB 2.0. I'm using Arch with kernel 3.2.27-8 and firmware 20120926.

Are there any newer kernels or firmware that I could try? If I can provide any debugging output that could help you, let me know.

By the way -- is there any reference page maybe with descriptions of the cmdline.txt parameters?

ghollingworth commented 11 years ago

What is the throughput you've expecting?

Gordon

On 27/09/2012, Ruben Jenster notifications@github.com wrote:

@ghollingworth The terratec cinergy htc stick is a USB 2.0 device

Reply to this email directly or view it on GitHub: https://github.com/raspberrypi/linux/issues/82#issuecomment-8929919

Sent from my mobile device

rmoszczynsk commented 11 years ago

A typical DVB-T stream in my location is about 24Mbit/s. Ideally I'd like to be able to process DVB-T2 streams, whose bitrates can exceed 30MBit/s, but I'm not sure if it's not too much for the hardware in RPI. Anyway, getting the 24Mbit/s to work would be nice for starters.

rmoszczynsk commented 11 years ago

BTW, I'm not trying to decode the streams -- just to record them, which I believe is a pure I/O excersise in which the CPU should not have a big impact (but I might be wrong here). I guess recording 24Mbit/s to an SD card isn't likely to work, but hopefully an external USB 2.0 hard drive could do it. But I need to sort out the lost packets problem in the first place.

r10r commented 11 years ago

I think that should work if I'm not completely mistaken. Modern class 10 SDHC cards should make about 20 MByte/s. For me the DVB-C stream size is about 38Mbit/s which is about 4 MByte/s (38000/8/1000). USB 2.0 can do a maximum of 60MByte/s (48000/8/1000). So this should not be the problem.

wiredbob commented 11 years ago

Hi guys, great thread. Did anyone manage to identify a fix for isochronous transfers? I've got a PCTV Nanostick T2 and have the same issues with glitching of the transport stream as discussed above and would love to get it working.

Thanks.

ghollingworth commented 11 years ago

Does anyone have a link to the firmware for the PCTV Nanostick 73e I've got hold of one but don't have the firmware files...

rmoszczynsk commented 11 years ago

I believe this is what you need: http://linuxtv.org/downloads/firmware/dvb-usb-dib0700-1.20.fw

ghollingworth commented 11 years ago

Thanks,

Found it... Now trying to work out all the random tools to actually get it to output the dvb stream!

I'm sure I'll get there in the end but if anyone has a crib note I can use to get hold of a single channel...

Gordon

On 08/11/2012, grizlak notifications@github.com wrote:

I believe this is what you need: http://linuxtv.org/downloads/firmware/dvb-usb-dib0700-1.20.fw

Reply to this email directly or view it on GitHub: https://github.com/raspberrypi/linux/issues/82#issuecomment-10188761

Sent from my mobile device

rmoszczynsk commented 11 years ago

You might try dvbstream for starters. This command will grab the whole transport stream (all channels broadcasted on a single frequency) and record 180 seconds of it to a file. You'll be able to play out the file later e.g. in VLC.

dvbstream 8192 -f -n 180 -o > file.mpg

The frequency value is based on where you're located... You can Google around for the list of frequencies available in your area, or scan the whole frequency range with a tool called w_scan.

rmoszczynsk commented 11 years ago

The frequency comes after "-f", e.g. 650000

ghollingworth commented 11 years ago

OK, I ran a w_scan and it came up with the following channels...

T 498000000 8MHz 2/3 NONE QAM64 8k 1/32 NONE T 522000000 8MHz 2/3 NONE QAM64 8k 1/32 NONE T 690000000 8MHz 3/4 NONE QAM64 8k 1/32 NONE # Cambs & Beds T 714000000 8MHz 3/4 NONE QAM64 8k 1/32 NONE # Cambs & Beds T 722000000 8MHz 3/4 NONE QAM64 8k 1/32 NONE

I'm assuming then I need to do a dvbstream 8192 -f 722000000 -n 180 -o > file.mpg

It just gives me

"Not able to lock to the signal on the given frequency"

Might be that the signal isn't strong enough here (I'm just using the mini antenna supplied with the dongle so don't know how good the signal is going to be) What do you think? Do you think it should work correctly?

Thanks

Gordon

rmoszczynsk commented 11 years ago

One quick fix you might want to try is input the frequency in kHz instead of Hz, so instead of 722000000 use 722000. If this doesn't work, we'll try something different.

ghollingworth commented 11 years ago

Got it working using tzap and cat'ing dvr0 to a file... Don't get any error messages yet though

Gordon

On 08/11/2012, grizlak notifications@github.com wrote:

One quick fix you might want to try is input the frequency in kHz instead of Hz, so instead of 722000000 use 722000. If this doesn't work, we'll try something different.

Reply to this email directly or view it on GitHub: https://github.com/raspberrypi/linux/issues/82#issuecomment-10195860

Sent from my mobile device

rmoszczynsk commented 11 years ago

It probably won't give you any error messages at this point. You could try and open the resulting file in VLC and look at Tools -> Codec information -> Statistics to see how many corrupted/discontinued frames you get. Or even better, run it through a proper analyzer (http://www.tsreader.com/tsreader/index.html, Windows only) and see how many continuity errors it shows.

wiredbob commented 11 years ago

Yep, you won't see any errors writing the file with tzap. Try playing back the ts with mplayer and you'll see the stats in the console -

e.g.

mpeg2video @ 0x7ffaf07264c0]concealing 45 DC, 45 AC, 45 MV errors A:28462.2 V:28462.2 A-V: 0.008 ct: -0.280 173/173 10% 36% 0.7% 0 0 [mp2float @ 0x7ffaf07264c0]Header missing [mpeg2video @ 0x7ffaf07264c0]00 motion_type at 8 29 [mpeg2video @ 0x7ffaf07264c0]concealing 45 DC, 45 AC, 45 MV errors A:28462.5 V:28462.5 A-V: 0.033 ct: -0.263 179/179 10% 36% 0.7% 0 0 [mpeg2video @ 0x7ffaf07264c0]00 motion_type at 31 6 [mpeg2video @ 0x7ffaf07264c0]concealing 0 DC, 0 AC, 0 MV errors A:28462.9 V:28462.9 A-V: 0.018 ct: -0.239 189/189 10% 36% 0.7% 0 0 [mpeg2video @ 0x7ffaf07264c0]00 motion_type at 12 18 [mpeg2video @ 0x7ffaf07264c0]concealing 45 DC, 45 AC, 45 MV errors A:28463.0 V:28463.0 A-V: 0.017 ct: -0.234 192/192 10% 37% 0.7% 0 0 [mpeg2video @ 0x7ffaf07264c0]ac-tex damaged at 10 15 [mpeg2video @ 0x7ffaf07264c0]concealing 45 DC, 45 AC, 45 MV errors A:28463.0 V:28463.0 A-V: 0.012 ct: -0.233 193/193 10% 37% 0.7% 0 0

ghollingworth commented 11 years ago

So I just captured a few seconds of video from the pi and simultaneously captured through my usb analyser and then wrote a nice little application to track the pids and output when the continuity count fails...

The only drops I find occurred in the device...

I've also noticed that the PCTV nanoStick 73E I've got seems to have some strange errors in its transmission. I get data length errors... Does anyone know if this has been seen before somewhere, couldn't find any reference to it on google...

Gordon

ghollingworth commented 11 years ago

I recorded about a minute of ITV1 using the 73e and had no errors in the stream.

Is there anyone who had trouble streaming from this device?

Gordon

On 08/11/2012, wiredbob notifications@github.com wrote:

Yep, you won't see any errors writing the file with tzap. Try playing back the ts with mplayer and you'll see the stats in the console -

e.g.

mpeg2video @ 0x7ffaf07264c0]concealing 45 DC, 45 AC, 45 MV errors A:28462.2 V:28462.2 A-V: 0.008 ct: -0.280 173/173 10% 36% 0.7% 0 0 [mp2float @ 0x7ffaf07264c0]Header missing [mpeg2video @ 0x7ffaf07264c0]00 motion_type at 8 29 [mpeg2video @ 0x7ffaf07264c0]concealing 45 DC, 45 AC, 45 MV errors A:28462.5 V:28462.5 A-V: 0.033 ct: -0.263 179/179 10% 36% 0.7% 0 0 [mpeg2video @ 0x7ffaf07264c0]00 motion_type at 31 6 [mpeg2video @ 0x7ffaf07264c0]concealing 0 DC, 0 AC, 0 MV errors A:28462.9 V:28462.9 A-V: 0.018 ct: -0.239 189/189 10% 36% 0.7% 0 0 [mpeg2video @ 0x7ffaf07264c0]00 motion_type at 12 18 [mpeg2video @ 0x7ffaf07264c0]concealing 45 DC, 45 AC, 45 MV errors A:28463.0 V:28463.0 A-V: 0.017 ct: -0.234 192/192 10% 37% 0.7% 0 0 [mpeg2video @ 0x7ffaf07264c0]ac-tex damaged at 10 15 [mpeg2video @ 0x7ffaf07264c0]concealing 45 DC, 45 AC, 45 MV errors A:28463.0 V:28463.0 A-V: 0.012 ct: -0.233 193/193 10% 37% 0.7% 0 0

Reply to this email directly or view it on GitHub: https://github.com/raspberrypi/linux/issues/82#issuecomment-10208633

Sent from my mobile device

rmoszczynsk commented 11 years ago

I've never used 73e. But I'm curious if you would get an error-free stream if you tried to record the whole multiplex (dvbstream with dummy PID 8192).

ghollingworth commented 11 years ago

I never got dvbstream to work, it just failed to lock no matter what I gave it...

Gordon

On 12 November 2012 12:09, grizlak notifications@github.com wrote:

I've never used 73e. But I'm curious if you would get an error-free stream if you tried to record the whole multiplex (dvbstream with dummy PID 8192).

— Reply to this email directly or view it on GitHubhttps://github.com/raspberrypi/linux/issues/82#issuecomment-10285062.

simonhorlick commented 11 years ago

I am having the same problem getting lock on a 290e. How did you take the ITV sample?

ghollingworth commented 11 years ago

I used x_scan to output a channels.conf file

You then need to copy this into somewhere like ~/.tzap/something

Then tune to the ITV1 channel (-r creates a streaming output) (you can use -S to shut it up tzap -r "ITV1"

then while tzap is still running:

cat /dev/dvb/adapter0/dvr0 > file.mpg

Gordon

rmoszczynsk commented 11 years ago

About dvbstream not being able to lock: I'm not sure, but 290e has two frontends (one for DVB-T/T2 and one for DVB-C). Maybe 73e is the same and dvbstream was choosing the wrong frontend by default?

hadjedjvincent commented 11 years ago

The no-lock problem may be a power one (my 460e stick was not able to lock without being connected to a powered hub). Anyway, DVB-S, DVB-S2 ou DVB-C is still not working properly here (lot of packet loss, count depends of stream bandwidth)

licaon-kter commented 11 years ago

This happens with kernel 3.6.y too?

rmoszczynsk commented 11 years ago

I did some tests last week with kernel 3.6.11 and PCTV Nanostick 290e. Same results as with 3.2.27: packets lost with DVB-T.

ghollingworth commented 11 years ago

What bitrate were you trying to download and what were you trying to do with it?

Were you just streaming it directly to a file?

Gordon

On 7 Jan 2013, at 07:40, grizlak notifications@github.com wrote:

I did some tests last week with kernel 3.6.11 and PCTV Nanostick 290e. Same results as with 3.2.27: packets lost with DVB-T.

— Reply to this email directly or view it on GitHub.

hadjedjvincent commented 11 years ago

It's the same if i'm streaming with VDR or TVHeadend or using XBMC via Live TV. But some channels like BFM TV on Astra 19.2E aren't getting so much artefacts (but the bandwith is not high... so...)

Regards,