search

mpv-player / mpv

🎥 Command line video player
https://mpv.io
Other
28.54k stars 2.92k forks source link

bisected: DVB broken, not able to lock to the signal on the given frequency #4922

Closed bpaterni closed 7 years ago

bpaterni commented 7 years ago

bisected commit

e40c41de19667889da4458a6f5c61c7d46abdb1f

mpv version and platform

dvb tuner: Hauppauge WinTV-HVR2255

05:00.0 Multimedia controller [0480]: Philips Semiconductors SAA7164 [1131:7164] (rev 81)
    Subsystem: Hauppauge computer works Inc. SAA7164 [0070:f111]
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0, Cache Line Size: 64 bytes
    Interrupt: pin A routed to IRQ 64
    NUMA node: 0
    Region 0: Memory at fdc00000 (64-bit, non-prefetchable) [size=4M]
    Region 2: Memory at fd800000 (64-bit, non-prefetchable) [size=4M]
    Capabilities: <access denied>
    Kernel driver in use: saa7164
    Kernel modules: saa7164

mpv: 0.27.0 distribution: debian unstable kernel: 4.12.0-2-amd64 (4.12.13.1)

Reproduction steps

mpv dvb://<channel>

Expected behavior

Successful playback of DVB content

Actual behavior

mpv is unable to tune to the channel's signal. Instead it errors out, mentioning:

[dvbin] Not able to lock to the signal on the given frequency, timeout: 30

Log file

verbose log of dvb signal lock timeout: https://pastebin.com/5X1uA2CC

dmesg: https://pastebin.com/ZzUAGQK5

olifre commented 7 years ago

Hi,

I'm not sure I can help, I was only involved in (some of) the non-ATSC-related changes of that commit. Maybe first a few questions that could help pin things down, since one major change was that mpv now supports cards which offer multiple transports:

bpaterni commented 7 years ago

I think you are on the right track! Assuming over-the-air broadcast == ATSC and cable broadcast == DVB-C. The card only has one TV input, however I have the ability to switch between OTA and Cable sources with independent channels files for both. If I plug the OTA antenna in, mpv works. However if I plug cable in, signal lock timeouts.

To generate the channels configs, I use the scan utility with 'legacy' tuning data files. Perhaps I'm not well informed, or for some other reason, I'm unable to generate channels.conf via dvbscan or using newer tuning data files together with scan.

For OTA channels: scan /usr/share/dvb/dvb-legacy/atsc/us-ATSC-center-frequencies-8VSB

For Cable channels: scan /usr/share/dvb/dvb-legacy/atsc/us-Cable-Standard-center-frequencies-QAM256

The channels generated for Cable are located at: https://pastebin.com/Nxmr8wUQ

olifre commented 7 years ago

Ahhh - you are in North America. Then you are indeed in a "ATSC-only" region, i.e. there should be no DVB-C. The difference between OTA and Cable, which are both ATSC I think, is only the modulation.

This means mpv is doing right in assuming ATSC is the way to go - still, the tuning fails. From the verbose log, however, I see mpv is using QAM_256 to tune, which would match with the "cable" assumption, so this seems right.

My problem is I don't have access to any ATSC broadcast (I'm in Europe here) nor to an ATSC-capable card, so I can only guess what could still be wrong. From the code, I only see one effect of the commit on the tuning for ATSC: Before that commit, the old kernel API (DVB-API v3) was used, while now it uses DVB-API v5, but it appears to pass through the same options (frequency, modulation etc.). I guess since you did bisect, you are familiar to compiling mpv yourself? If so, could you try to modify stream/dvbin.h and change:

#define DVB_USE_S2API 1

to 

#define DVB_USE_S2API 0

recompile, retry and attach a verbose log again? This will make mpv unaware of your kernel being modern enough to support DVB-API v5, so it will compile with DVB-API v3 support only. If this works, at least we know we are in the right place (and if not, maybe the verbose log will tell us something new).

Cheers and many thanks!

olifre commented 7 years ago

Ah, and on a side-note: I think you can use w_scan to create new initial tuning data (and to create channels.conf without initial data). But I am not sure what are the correct options for ATSC, but it seems your channel.conf is fine, so that's not the issue.

bpaterni commented 7 years ago

Ok, thank you very much for the above information! Turns out instead i needed to comment out

//#define DVB_USE_S2API 1

entirely since the conditional checks for it seem to be checking whether it is defined and not the value of it. With it commented out, it results in a working stream! So it looks like somehow S2API has broken ATSC DVB streams(?)

My mpv log above was produced via the latest 0.27.0 release, so I've checked out code at commit e40c41de19667889da4458a6f5c61c7d46abdb1f and produced two more logs for better comparison:

log.e40c41de19 log.e40c41de19.no_use_s2api

olifre commented 7 years ago

Thanks for figuring out the correct way to "undefine", of course you are right, commenting out is the right way ;).

With it commented out, it results in a working stream! So it looks like somehow S2API has broken ATSC DVB streams(?)

Really seems like it - now, since I can not gather more information from the logs (but it looks like until the "tune" part everything is exactly the same), let's try to pin it down even more.

Could you start from plain e40c41d (i.e. with S2API on), and then modify https://github.com/mpv-player/mpv/blob/e40c41de19667889da4458a6f5c61c7d46abdb1f/stream/dvb_tune.c#L590 from

#ifdef DVB_USE_S2API

to

#if 0

? This should leave all S2API-calls active (tuner detection etc.), and revert to the old v3 API for the actual tuning only.

If this also results in a working mpv, then we are down to a few lines of code (basically a single ioctl). The problem is then to find out what is missing from the DVBv5 call. Maybe I could then have a look at the code of w_scan again then and see whether they do something special for ATSC there.

bpaterni commented 7 years ago

We're in luck olifre! It works when tuning via DVB-API v3!

log.e40c41de19.tune.via.dvbapi.v3

olifre commented 7 years ago

Perfect! :) Since it's night by now here in Europe, I will only be able to investigate further tomorrow, but that pins it down significantly!

olifre commented 7 years ago

Could you try whether w_scan (it's packaged for Debian in the package w-scan) works for your card? It also uses DVBv5 API internally, so if it works, I can check whether they do something differently in the tuning stage. How to use w_scan for ATSC aerial and cable is described here: https://linuxtv.org/wiki/index.php/W_scan#Examples The nice thing is that w_scan can also produce initial data for scanning, and perform scans by itself without initial data ;).

bpaterni commented 7 years ago

Sorry for the delay, was away from the tuner for some time.

w_scan is able to find channels available and produce a working channels.conf via DVB-API-v3 mpv!

For reference, the command used:

$ w_scan -fa -A2 -c US -X >> channels.conf

VDR output did not seem to work in mpv, so -X was necessary for zap output

olifre commented 7 years ago

VDR output did not seem to work in mpv, so -X was necessary for zap output

I think that was never implemented for ATSC. My original VDR-format implementation was for DVB-S / DVB-S2 only, since for those dual-transport devices, the zap format does not contain sufficient information. I think it was extended to any DVB-format in the meanwhile.

Sorry for the delay, was away from the tuner for some time.

No problem! Now I just need to compare the way the two codes tune - and spot the difference. I'll let you know once I'm onto something! 👍

olifre commented 7 years ago

Hi!

Could you please test whether: https://github.com/olifre/mpv/commit/905116170d94597f61cf31afffddc75d3c641338 helps? I did several changes:

If it works with my changes, please attach a verbose log again so I can see whether the code worked as expected, or the new fallback path was used. I'll keep my fingers crossed it works - for DVB-S / DVB-S2 the new code works fine, and the tuning code was even simplified for this delivery system. I also hope this won't break DVB-T / DVB-T2 (which now uses more information).

olifre commented 7 years ago

For future reference, this is the piece of code w_scan is using: https://github.com/stefantalpalaru/w_scan2/blob/9f0c2ae8c3fee4284346de453aa70c49f71134af/src/scan.c#L2445

bpaterni commented 7 years ago

hrm... That is strange! Now neither v5 or the fallback to v3 work :(

log.905116170d

olifre commented 7 years ago

hrm... That is strange! Now neither v5 or the fallback to v3 work :(

That is indeed very weird - I did not change anything in the code which we fallback to... Could you try again to modify (in the modified version with my patch applied you have now):

https://github.com/olifre/mpv/blob/905116170d94597f61cf31afffddc75d3c641338/stream/dvb_tune.c#L590

from 

#ifdef DVB_USE_S2API

to

#if 0

and see whether that fixes things? If so, this would mean that somehow the DVBv5 tuning command somehow leaves the card / kernel driver in a strange state. Does w_scan still work in this situation?

In any case, that's really strange, since now I can't spot any difference anymore (for the ATSC code) as compared to w_scan...

bpaterni commented 7 years ago

Disabling DVBv5 tuning does result in a working stream, and w_scan does still work following an unsuccessful tune using the new v5 code. From only glancing at the code, I see that the fallback to v3 is only a goto statement. My only guess would be something in the v5 code might be modifying the inputs to the v3 code(?)

Aside from that though, I've got a new issue: opengl video output no longer works and i had to use --vo tct to test. In the verbose log it mentions this:

[vo/gpu] Probing for best GPU context.
[vo/gpu/opengl] Initializing GPU context 'drm'
[vo/gpu/opengl] DRM EGL backend can be activated only manually.
[cplayer] Error opening/initializing the selected video_out (--vo) device.

Maybe it has something to do with a recent mesa upgrade from debian?

olifre commented 7 years ago

Disabling DVBv5 tuning does result in a working stream, and w_scan does still work following an unsuccessful tune using the new v5 code.

Then my only explanation would be that the DTV_CLEAR that's issued before DVBv5 tuning by w_scan fixes the situation (mpv also does that, but of course does not do that before DVBv3 tuning). I cross-checked the DVBv5-tuning code again, there are no variables being touched which could affect DVBv3 (all magic happens with local variables). I added another patch on top of the old one: https://github.com/olifre/mpv/commit/105401db7500f48d92e8f3f0cd083e9600d232d5 which explicitly does a DTV_CLEAR before entering the DVBv3 fallback. Could you test whether that at least fixes the fallback?

Sadly, this still leave the issue open why the tuning using DVBv5 itself does not work. The only remaining idea I have would be to add more output, i.e. dump the full tuning command in mpv and do the very same thing in w_scan. I could prepare a patch against w_scan and another patch for mpv to do just that, but then of course that means you'd also have to compile your own `w_scan' for this test.

Maybe it has something to do with a recent mesa upgrade from debian?

Mhhhm - it's a bit strange it tries to probe DRM only. I think it's best to open a separate issue here on this, since I am no expert on the vo-selection logic, but many others are ;).

bpaterni commented 7 years ago

The explicit clear does not fix the DVBv3 fallback! Oo

olifre commented 7 years ago

The explicit clear does not fix the DVBv3 fallback! Oo

Things are getting stranger and stranger... Just to exclude I am missing anything... Could you please test whether changing:

            if (ioctl(fd_frontend, FE_SET_PROPERTY, &cmdseq) < 0) {
                MP_ERR(priv, "ERROR tuning channel\n");
                goto old_api;
            }

to

goto old_api;

in https://github.com/olifre/mpv/blob/105401db7500f48d92e8f3f0cd083e9600d232d5/stream/dvb_tune.c#L706 i.e. removing the ioctl changes anything? That would be the most un-invasive change removing DVBv5 tuning for ATSC only.

bpaterni commented 7 years ago

Well that does it! Don't do the ioctl and fallback to DVBv3 works! :)

olifre commented 7 years ago

Finally at last... So this really means the ‘ioctl’ which is the failing tune command upsets the device (or kernel driver). So as next step (tomorrow), I'll add some lines to dump the actual full ’ioctl’ parameter set, and the same for ’w_scan’. Something must be off with the actual values used for tuning...

olifre commented 7 years ago

Hi! The part inside mpv is done: https://github.com/olifre/mpv/commit/8bdcd9c9cf636dc725ffa22b42d2b3a14cde1dbc After applying this patch, your verbose log should contain a line reading Dumping raw tuning commands and values: followed by a series of funny numbers in the following lines.

Now I'll have a look at w_scan and see how to implement it there with a patch - I'll do a quick-and-dirty patch there sufficient for our debugging.

olifre commented 7 years ago

And here we go for w_scan: https://gist.github.com/olifre/06e34d8ba4c28fb267142d74371323ac I made this patch against the current w_scan-20170107 tarball you can get from their homepage at http://wirbel.htpc-forum.de/w_scan/index2.html . It should make w_scan throw a few lines for each single tuning attempt on the terminal - if it's really long, maybe I can also learn something from a small snippet of the output already.

bpaterni commented 7 years ago

Ok! It does look like there's a slight difference between mpv/w_scan tuning, though I couldn't describe what it means:

mpv.log.8bdcd9c9cf:

[   0.472][v][dvbin] Dumping raw tuning commands and values:
[   0.472][v][dvbin] 00: 0x11(17) => 0xb(11)
[   0.472][v][dvbin] 01: 0x3(3) => 0xcb21f40(213000000)
[   0.472][v][dvbin] 02: 0x6(6) => 0x2(2)
[   0.472][v][dvbin] 03: 0x4(4) => 0x5(5)
[   0.472][v][dvbin] 04: 0x1(1) => 0x0(0)

w_scan.log:

Dumping raw tuning commands and values:
00: 0x2(2) => 0x0(0)
01: 0x11(17) => 0x2(2)
02: 0x3(3) => 0xcb21f40(213000000)
03: 0x6(6) => 0x2(2)
04: 0x4(4) => 0x5(5)
05: 0x1(1) => 0x0(0)
olifre commented 7 years ago

Ok! It does look like there's a slight difference between mpv/w_scan tuning, though I couldn't describe what it means:

There's even a huge difference! Thanks for testing this through!

To explain: The first line which w_scan does in addition is the "clear" call, which mpv also does, but before the tuning. So that's effectively not a difference, it's just not included in the output.

The major issue is the "0x11" setting, which is the choice of "delivery system". This shows that mpv is switching to "0xb", which translates into ATSC. However, w_scan is using "0x2", which is "DVB-C ANNEX B"! Up to now, I thought in the US there was ATSC only for cable and aerial reception, but apparently there is also DVB-C for cable in addition. Also, your channels.conf was in ATSC format...

This also explains the other strangeness else we have seen: If mpv tunes via DVBv5, it explicitly selects ATSC. Since the DVBv3 fallback can not select a delivery system, it also has to fail, since it will try to use ATSC for tuning. If mpv never tries DVBv5, the card seems to prefer DVB-C, which is why it works if you disable the DVBv5 code completely. That's also why old mpv versions worked (they could not choose the delivery system).

This also tells me what the solution must be: mpv needs a different channel.conf format. For DVB-C, it also needs it know other stuff (inversion, samplerate, forward error correction setting).

So my job is now to re-read the code to see which exact format is needed, and check how to generate that (e.g. with w_scan). I'll let you know, hopefully tomorrow - at least we are a huge step further now!

olifre commented 7 years ago

Hi! Sorry it took so long. I didn't have much time yesterday, but now I finally came around checking the cable-tuning-format.

What mpv expects should match the output w_scan creates in "xine"-mode, i.e. with the -X option. Could you test whether this creates a working channel-config for you? Additionally, I did not find any clear explanation why mpv should not work with the vdr-style output for cable tuning. So it would be cool if you could also test creating a vdr-style config again (best with w_scan) and get a verbose log from mpv, maybe I can see what's going wrong ;-).

Many thanks in advance!

bpaterni commented 7 years ago

I believe 'xine'-formatted channels have been all I've been using so far whether created via scan or w_scan.

Nonetheless, I've generated new channels files with w_scan for both and tested them out again with your unmodified dvb-v5-tuning-fixes branch

$ w_scan -fa -A2 -c US -X >> channels.conf.ics.xine channels.conf.ics.xine mpv log with xine channels

$ w_scan -fa -A2 -c US >> channels.conf.ics.vdr channels.conf.ics.vdr mpv log with vdr channels

With xine channels, mpv exhibits the same behavior seen throughout this issue: tuning is attempted with dvbv5, locking fails, and then DVBv3 is attempted, but fails there too. Curiously with vdr channels though, both v5 and v3 fail immediately with no pause waiting for a timeout.

olifre commented 7 years ago

Mhhhm... something does not match up here... As found before, w_scan tunes using 0x11(17) => 0x2(2), i.e. DVB-C Annex B. However, you explicitly tell it it to tune ATSC, and the channels-config it creates is of ATSC format. So I had a short look at more code in w_scan again. It seems the scan type it uses internally is called SCAN_TERRCABLE_ATSC, while if you give it -A2, it sets the modulation to QAM. With a funny function called atsc_del_sys(fe_modulation_t modulation) it then later chooses the actual delivery system used for tuning, which is SYS_DVBC_ANNEX_B for for QAM modulation.

From this, I deduce that ATSC cable in reality means SYS_DVBC_ANNEX_B. Checking the code for "cable" tuning in w_scan, that uses SYS_DVBC_ANNEX_A.

The VDR part fails for an additional reason: Deducing from w_scan code, it seems the frequency values are divided by 1000 (i.e. so we get kHz) for all delivery systems - mpv multiplies it with 1000 again, but only for DVB-S / DVB-S2 (which was the only code I could test myself, and the format is sparsely documented).

That means I need to fix two things:

olifre commented 7 years ago

Mhhhm... Could you try changing: https://github.com/olifre/mpv/blob/8bdcd9c9cf636dc725ffa22b42d2b3a14cde1dbc/stream/dvb_tune.c#L155 Just remove the line-pair

#if 0 /* Not used now. */
...
#endif

such that the code in between becomes active? I'm not sure why this was left out, this was in a change not done by me, I'll try to find out ;-).

In any case, I don't expect that this will really fix it - this is in the DVBv3-only part. The fix needs to go at a different place...

olifre commented 7 years ago

Ok, I was faster than expected... Could you try whether the newest version of my https://github.com/olifre/mpv/commits/dvb-v5-tuning-fixes branch fixes two things?

I can fix the latter point likely tomorrow - for this, it's really helpful to have your channels config, many thanks! :-)

An additional piece of information I found while reading the code: If it would somehow be possible to connected the card to cable and terrestrial at the same time (maybe there are cards with two inputs?), I think -A3 should work as parameter to w_scan. The way I implemented things in mpv, it should do the correct thing "channel by channel" (i.e. cable or terrestrial tuning).

bpaterni commented 7 years ago

Something different is happening now for sure, but it still does not result in a working stream :(

For xine channels, mpv looks like it advances pass the tune ioctl, but is unable to read data: log.4422983626c90f2d943df3e55b762ff14d7b252c.xine

For vdr channels, mpv also seems to get pass tuning, but then hangs: log.4422983626c90f2d943df3e55b762ff14d7b252c.vdr

[   0.398][v][dvbin] Unknown FE type. Aborting
[   0.398][v][dvbin] DVB_SET_CHANNEL: PMT-PID for service 1 not resolved yet, parsing PAT...

An additional piece of information I found while reading the code: If it would somehow be possible to connected the card to cable and terrestrial at the same time (maybe there are cards with two inputs?), I think -A3 should work as parameter to w_scan. The way I implemented things in mpv, it should do the correct thing "channel by channel" (i.e. cable or terrestrial tuning).

Unfortunately, the tuner card I have now is limited to 1 physical input, but that input somehow supports duplexing, allowing two stations to be tuned at the same time. Though I do wish there was a way to combine both cable and terrestrial signals to be fed into single tuner input! That would prevent having to manually switch the input cable!

olifre commented 7 years ago

Something different is happening now for sure, but it still does not result in a working stream :(

My bad! I adapted all places, apart from the one right before tuning... I added the missing one-liner just now to my branch, if you retry, xine should work (I hope :-) ) and for VDR a fix is upcoming later today.

I do wish there was a way to combine both cable and terrestrial signals to be fed into single tuner input!

I'm not sure whether the frequencies for ATSC terrestrial and ATSC cable (DVBC) overlap - the channel-config for both scans should tell. If they don't overlap, it should be possible to have some box in between which merges the two cables into one ;-).

bpaterni commented 7 years ago

That did it olifre! At least for xine channels: log.8319d99ce8cb5e4598bce164063d0177e9dff957.xine.works

olifre commented 7 years ago

That did it olifre! At least for xine channels:

Perfect! :) I am still working on the VDR part, at least from the w_scan code the syntax is pretty straightforward, but the parsing is a bit ugly. I will only add the parsing of "modulation", which should be sufficient for ATSC cable. For DVB-T, a lot more would be needed, I'll leave that for some other time when a user requests it ;-).

olifre commented 7 years ago

Done! My branch should now also work with VDR-style config files for ATSC terrestrial / cable. On a side-note, this means the modulation is also parsed for other delivery systems, e.g. DVB-S/S2, where it is not strictly required, but should help tuning faster ;-).

bpaterni commented 7 years ago

Err, crap! Something might have goofed up. Now neither xine/vdr are functioning:

log.b7ec266fe6001fa621af951407ea3406bb1655d5.vdr

[   0.060][e][dvbin] DVB CONFIGURATION IS EMPTY, exit
[   0.060][e][stream] Failed to open dvb://WHO.
olifre commented 7 years ago

Argh! That was caused by a protection I inserted: If the delivery system is not supported, the channel config for it is skipped. Since "DVBC_ANNEX_B" was still missing from the global mask of supported systems, this caused all channels to be skipped. I commited a fix just now, could you please test again?

bpaterni commented 7 years ago

That does it! VDR channels work as well now!

log.86ba41e7c8abf8d4e672363c8752a94101037bf5.vdr

Thanks so much olifre! You are the most awesomenest! :)

olifre commented 7 years ago

Perfect :-) Many thanks for your detailed testing, without the actual hardware (and broadcasting around here), I would never have been able to test things ;-). Now I'll open a PR and try to get the changes into mpv so they enter the next release :).

olifre commented 7 years ago

@bpaterni The pull request is merged, so this should enter a future mpv release (and you can now build mpv from the official master branch if you want ;-) ). Many thanks for all your testing!