Mythtv MPEG-TS demux fails on playback of YLE recordings

[kmdewaal]

• Platform: Fedora 33

• MythTV version: MythTV master

• Package version: Built from source

• Component:

What steps will reproduce the bug?

There are various issues with playback of recordings of the Finnish YLE channel, such as a display of a 640x480 window from the top-left corner of a 1920x1080 video stream and excessive delay before starting playback. This problem has been analyzed in ticket trac-13557 "Decode errors" https://code.mythtv.org/trac/ticket/13557 (thanks to JPilk for pointing this out) and also recently in the forum in topic "HD Playback problem with YLE-channels only" https://forum.mythtv.org/viewtopic.php?p=21615#p21615.

The issue seems to occur mainly when there is a scene change, for example from a leader to the real program. Note that VLC and ffmpeg do play the recordings correct.

There is an mythfrontend option to "Use FFmpeg's original MPEG-TS demuxer". When this option is selected the playback in mythfrontend is correct. This option is currently only available when mythtv is built with "--compile-type=debug".

Thanks to JYA for shining some light on this issue with the following:

I added that option many moons ago.
It was mostly designed at the time to check if a mpeg-ts file was
broken; not something that would typically be a problem with OTA
recordings but when dealing with HLS streaming could be quite common.

The mythtv mpeg-ts demuxer is very old and has received very little
updates over the years.

Things that will be broken using the ffmpeg's mpegts demuxer are
mostly subtitles and how to choose different audio tracks.
It doesn't support the callbacks that myth required when the
resolution or the content changes.

So it's fine to play a simple file, anymore than that and it's not gonna work.

With all this information the conclusion is that the MPEG-TS demuxer is the component that needs to be fixed. This will be a non-trivial effort so in the meantime the plan is to make the "Use FFmpeg's original MPEG-TS demuxer" option unconditionally available for both master and v31 so that it is also in the packages. A short test shows that audio and subtitles are detected correct at least initially so using this option is a viable workaround for the playback of YLE recordings.

In the original recording the place in the file that causes the issues is between megabytes 68 and 69. The part of that file from megabyte 65 to megabyte 75 is attached and with this clip the problem can be demonstrated. N.B. The .mov extension has been added to allow upload to Github.

https://user-images.githubusercontent.com/10491032/113342089-53e18780-932e-11eb-8ca7-5bbabb476aae.mov

How often does it reproduce? Is there a required condition?

What is the expected behaviour?

What do you see instead?

Additional information

[bennettpeter]

In the years since MythTV customized the TS demuxer, FFmpeg themselves may have added some of the features we need. I believe that USA subtitles may be one of those things that was not originally supported.

[kmdewaal]

I did recently look at the USA subtitles and they are detected and displayed correct with the FFmpeg demux selected, at least on my test clips. But there may be other issues. Now that the option is generally available it will be easy to test if it there are issues with it. Of course, we should end up with having only one perfect demux.

[paul-h]

From what Jean-Yves said you will likely see problems where the stream changes like when streams are added or removed or changed. For example like when the adverts are in stereo and the the program is in surround sound or the resolution/aspect ratio changes or when you record part time channels that show some interactive screen before the channel starts then the audio and video streams only start when the program proper starts.

It is possible the newer TS demuxer available in current FFmpeg already handles some or all these problems but they are the sort of problems I would look out for.

[Jpilk]

I tried to download the video, but neither Firefox nor Google Chrome have cooperated. "No video with supported format and MIME type found". Fedora 33 https://bugzilla.mozilla.org/show_bug.cgi?id=1695173 The upload quoted on the forum has expired...

[Jpilk]

OK, I have it using 'curl -o YLEvid.ts https://user-images.githubusercontent.com/10491032/113342089-53e18780-932e-11eb-8ca7-5bbabb476aae.mov' and will play with it.

But I still suggest rescheduling the recording to start later, thereby avoiding the problematic intro.

[bennettpeter]

Running the clip with -v playback shows some strange things. I have my video playback set to NVDEC but this actually plays with ffmpeg. It starts off correctly with NVDEC NVDEC: NVDEC supports decoding 'H264 Main yuv420p Depth:8 1920x1080' Shortly thereafter, before even showing a single frame, it detects a change to resolution of 0x0 NVDEC: NVDEC does NOT support 'H264 Depth:8 0x0' Since NVDEC does not support resolution of 0x0 it switches to ffmpeg. Then it thinks it is 640x480. VidOutGPU: Video changed: 1920x1080 (1920x1080) 'H.264 NVDEC' (Aspect 1.77778 Refs 4)-> 640x480 (640x480) 'H.264' (Aspect 1.33333 Refs 16) However, tsanalyze finds nothing like this. With another file that actually changes resolution, I see tsanalyze showing both resolutions in the pids analysis report, but this sample only shows the one resolution, 1920x1080.

[kmdewaal]

Yes, initially the video is correctly detected. Then mpegts-mythtv.c detects a pmt change and on this change it flushes everything and starts again. In the second start it cannot find the video size anymore. The pmt change in this case is the adding and removing of a second AC3 audio stream. The comparable code in the ffmpeg mpegts.c does not do anything drastic when the pcrpid does not change. Or so it looks. It is difficult to compare mpegts-mythtv.c and mpegts.c because mpegts.c has also been developed a lot, with support for additional stream types for example. On first sight there is code in mpegts-mythtv.c added to support BBC IPTV streams, or actually bugs in that stream, that is not found in the ffmpeg mpegts.c. It could just be that this fix is not really relevant anymore, mpegts-mythtv.c has been created in 2011. I am considering to recreate the diff from 2011 and then see what can be applied to the latest ffmpeg demux and make that the new mpegts-mythtv.c. For this I would appreciate feedback on what actually goes wrong when using the ffmpeg demux.

[kmdewaal]

Difference found: mpegts-mythtv does pickup the second audio stream when it appears; ffmpeg-mpegts does not.

The critical point is where the second audio stream is added. This happens after megabyte 68 in the original recording (the attached clip is from megabytes 65 to 75) which is at time 1:48 according to the stream analyzer. When the clip is started from the beginning both mpegts-mythtv and ffmpeg-mpegts show one audio stream. This is OK. When the 1:48 time is passed then mpegts-mythtv does pickup the second audio stream, as shown in the menu. The ffmpeg-mpegts does NOT pickup the second audio stream. With the first 70 megabytes cut off then ffmpeg-mpegts also sees the second audio stream.

[bennettpeter]

I am interested in getting a bigger clip. The one you supplied is only a few seconds long and I think that is too short to see the audio change. You can use dropbox or google drive and paste a URL here. You won't have to rename the file to .mov.

[Jpilk]

The 10 MB clip now plays in new-ffmpeg master from start to end using both 'Normal' and nvdec decoding; but the aspect ratio looks like 4:3 with black bands at the side. From el7 via DLNA it plays fine, filling the TV screen. From Fedora 33 DLNA I get 'cannot read file' - but it's good (and loud) with leanfront.

Unfortunately the DB has moved beyond my Android mythfrontend again...

[kmdewaal]

This is the complete YLE recording: https://drive.google.com/file/d/1FuTWNeiIWT4chQdT2h5QucAIXeLrBHUr/view?usp=sharing

[Jpilk]

I thought at first that mythfrontend was playing different shows than SMPlayer and VLC - but as reported last year, mythtv is showing just the top left corner of the picture and I haven't found a way to skip, jump or get the current time mark. The other players can do this. mythcommflag --rebuild was slow (around 112fps) and didn't create a working seektable.

The 'dd skip 80MB' trick (that isn't the actual command) gives me good basic playback but still no I-frames, skipping or seektable.

[kmdewaal]

You can cut off the first 80 megabyte with dd and then the remainder plays fine. This is the command: dd if=yle-2002120210327155800.ts of=yle-skip80.ts bs=1024k skip=80 I got slightly confused with my configuration..... it plays fine, including seek etc, but only with the ffmpeg-tsdemux.....

[kmdewaal]

We are looking at a transport stream that is created by the MythTV recording system. It is possible that something goes wrong there. To really understand what is happening we need a capture of the full transport stream.

[Jpilk]

I now have the 'original ffmpeg decoder' selected in Fedora 33. Maybe these stream analyses could be useful.file:///home

/john/YLE_202104_probe.txt

[Jpilk]

YLE_202104_probe.txt

[Jpilk]

I didn't really look at that file before sending it. ffprobe looks informative, mythffprobe gives mainly zeroes ???

[Jpilk]

It looks as if mythffprobe is not extracting timing information from this file, which is a dd-like copy of a section of the clip.

YLE_202104_ffprobe.txt

[Jpilk]

I thought I had posted a comment similar to this yesterday, but perhaps I forgot the final 'Comment' step before a reboot.

I have a script, mainly intended for cutting, that leaves only one video and one audio stream. When using the distro's ffmpeg it works on the 'skip 79 MB' recording, and the result plays well. When using mythffmpeg it fails. Here are some details from yesterday's runs.

YLE_202104_to_video.txt

[kmdewaal]

The result of a remux with ffmpeg like this: ffmpeg -i input_file -c copy output_file plays OK with mpegts-mythtv. However, this leaves one video stream and one audio stream; it removes the additional audio stream and the subtitles.

When I do a remux with the VideoRedo application (Windows, commercial) I get the message that there are 13047 audio resync frames added. The additional audio stream and the subtitles are still there, although with two streams less. This also plays OK with mpegts-mythtv. Comparing log messages from mpegts-mythtv and ffmpeg-mpegts is difficult because there are many differences but there are audio-related messages in the mpegts-mythtv log that are not present in the ffpeg-mpegts log.

Tentative conclusion is that mpegts-mythtv is unhappy about the audio packets.

[kmdewaal]

Done some tests with a version of the clip in which the first 70 megabyte are skipped. This leaves a clip without scene changes that does not play correct with mpegts-mythtv as confirmed before. The log with ffmpeg-mpegts gives this:

[mpegts @ 00007fb7d11f0a40] Non-increasing DTS in stream 5: packet 48 with DTS 0, packet 49 with DTS 0
2021-04-07 18:54:19.295723 D [1204990/1204990] CoreContext decoders/avformatdecoder.cpp:285 (myth_av_log) - [h264 @ 00007fb7d0787a80] ct_type:0 pic_struct:1
2021-04-07 18:54:19.295752 D [1204990/1204990] CoreContext decoders/avformatdecoder.cpp:285 (myth_av_log) - [h264 @ 00007fb7d0787a80] ct_type:0 pic_struct:2
2021-04-07 18:54:19.295767 D [1204990/1204990] CoreContext decoders/avformatdecoder.cpp:285 (myth_av_log) - [mpegts @ 00007fb7d11f0a40] Non-increasing DTS in stream 5: packet 49 with DTS 0, packet 50 with DTS 0
2021-04-07 18:54:19.295780 D [1204990/1204990] CoreContext decoders/avformatdecoder.cpp:285 (myth_av_log) - [h264 @ 00007fb7d0787a80] ct_type:0 pic_struct:1
2021-04-07 18:54:19.295792 D [1204990/1204990] CoreContext decoders/avformatdecoder.cpp:285 (myth_av_log) - [mpegts @ 00007fb7d11f0a40] Continuity check failed for pid 210 expected 0 got 15
2021-04-07 18:54:19.295816 D [1204990/1204990] CoreContext decoders/avformatdecoder.cpp:285 (myth_av_log) - [h264 @ 00007fb7d0787a80] ct_type:0 pic_struct:2
2021-04-07 18:54:19.295830 D [1204990/1204990] CoreContext decoders/avformatdecoder.cpp:285 (myth_av_log) - [mpegts @ 00007fb7d11f0a40] Non-increasing DTS in stream 5: packet 50 with DTS 0, packet 51 with DTS 0
2021-04-07 18:54:19.295836 D [1204990/1204990] CoreContext decoders/avformatdecoder.cpp:285 (myth_av_log) - 

This shows that in this stream the DTS (decode time stamp) is zero in each packet.

The same log from ffmpeg-mpegts at position packet 49 gives this:

[h264 @ 00007fd3b2aa5a80] ct_type:0 pic_struct:1
2021-04-07 18:58:45.587875 D [1205331/1205331] CoreContext decoders/avformatdecoder.cpp:285 (myth_av_log) - [mpegts-ffmpeg @ 00007fd3b350ea40] pid=136 pes_code=0x1e0
2021-04-07 18:58:45.587887 D [1205331/1205331] CoreContext decoders/avformatdecoder.cpp:285 (myth_av_log) - [h264 @ 00007fd3b2aa5a80] ct_type:0 pic_struct:2
2021-04-07 18:58:45.587901 D [1205331/1205331] CoreContext decoders/avformatdecoder.cpp:285 (myth_av_log) - [mpegts-ffmpeg @ 00007fd3b350ea40] Non-increasing DTS in stream 5: packet 49 with DTS 4425699617, packet 50 with DTS 4425699617
2021-04-07 18:58:45.587906 D [1205331/1205331] CoreContext decoders/avformatdecoder.cpp:285 (myth_av_log) - [mpegts-ffmpeg @ 00007fd3b350ea40] pid=13ec pes_code=0x1bd
2021-04-07 18:58:45.587911 D [1205331/1205331] CoreContext decoders/avformatdecoder.cpp:285 (myth_av_log) - [mpegts-ffmpeg @ 00007fd3b350ea40] pid=136 pes_code=0x1e0
2021-04-07 18:58:45.587927 D [1205331/1205331] CoreContext decoders/avformatdecoder.cpp:285 (myth_av_log) - [h264 @ 00007fd3b2aa5a80] ct_type:0 pic_struct:1
2021-04-07 18:58:45.587940 D [1205331/1205331] CoreContext decoders/avformatdecoder.cpp:285 (myth_av_log) - [mpegts-ffmpeg @ 00007fd3b350ea40] Continuity check failed for pid 210 expected 0 got 15
2021-04-07 18:58:45.587944 D [1205331/1205331] CoreContext decoders/avformatdecoder.cpp:285 (myth_av_log) - 

Here the message is that the DTS is the same as in the previous packet. However, the DTS is filled in, it is not zero. What happens is that the TS packet is duplicated. In principle a duplicate packet should IMHO not be processed but that is done here and then we do not get a DTS increase for the duplicated packet. Which makes sense. When playing back with VLC there are lots of messages about duplicate TS packets so that is consistent.

This is something tangible that the mpegts-mythtv does wrong and the ffmpeg-mpegts does correct. Although I am not familiar with the code I think I'll have a go at doing a big diff/merge and re-create the mpegts-mythtv based on the latest ffmpeg-mpegts plus the mythtv modifications.

[Jpilk]

Hi Klaas: I hope it goes well. Here are a few more stream analyses of the long clip, at 4MB and 99MB in, by ffmpeg and mythffmpeg. As you noted, an extra audio stream has popped up in the main (News) programme. I'm intrigued to see its target audience, but perhaps more significant is that it changes the numbering (zero based) of the other (subtitle) streams. What were streams 2,3,4 become 3,4,5. I remember mythtranscode used to have difficulties with pop-up streams. Maybe it still does.

First the distro's (actually rpmfusion's) version:

[john@HPFed rec22]$ ffmpeg -i 20101_20210402094904.ts 2>&1 | grep -A 8 -B 3 Video Input #0, mpegts, from '20101_20210402094904.ts': Duration: 00:00:18.97, start: 49065.385467, bitrate: 4422 kb/s Program 1 Stream #0:0[0x136]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 50 tbr, 90k tbn, 50 tbc Stream #0:10x352: Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, fltp, 448 kb/s Stream #0:20x467: Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006) Stream #0:30x47e: Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006) (hearing impaired) Stream #0:40x13ec: Subtitle: dvb_teletext ([6][0][0][0] / 0x0006), 492x250 At least one output file must be specified

[john@HPFed rec22]$ ffmpeg -i 20101_20210402094999.ts 2>&1 | grep -A 8 -B 3 Video Input #0, mpegts, from '20101_20210402094999.ts': Duration: 00:00:17.10, start: 49219.099378, bitrate: 4906 kb/s Program 1 Stream #0:0[0x136]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 50 tbr, 90k tbn, 50 tbc Stream #0:10x352: Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, fltp, 448 kb/s Stream #0:20x355: Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, fltp, 192 kb/s (visual impaired) (descriptions) Stream #0:30x467: Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006) Stream #0:40x47e: Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006) (hearing impaired) Stream #0:50x13ec: Subtitle: dvb_teletext ([6][0][0][0] / 0x0006), 492x250 At least one output file must be specified

And using mythffmpeg:

[john@HPFed rec22]$ mythffmpeg -i 20101_20210402094904.ts 2>&1 | grep -A 8 -B 3 Video [mpegts @ 0x56350747cb80] Packet corrupt (stream = 4, dts = 0). Input #0, mpegts, from '20101_20210402094904.ts': Duration: 00:00:18.80, start: 49065.552100, bitrate: 4461 kb/s Stream #0:0[0x136]: Video: h264 (Main), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 50 tbr, 90k tbn, 50 tbc Stream #0:10x352: Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, fltp, 448 kb/s Stream #0:20x467: Subtitle: dvb_subtitle Stream #0:30x47e: Subtitle: dvb_subtitle Stream #0:40x13ec: Subtitle: dvb_teletext At least one output file must be specified

[john@HPFed rec22]$ mythffmpeg -i 20101_20210402094999.ts 2>&1 | grep -A 8 -B 3 Video [mpegts @ 0x560d52178b80] Packet corrupt (stream = 5, dts = 0). Input #0, mpegts, from '20101_20210402094999.ts': Duration: 00:00:17.04, start: 49219.152100, bitrate: 4922 kb/s Stream #0:0[0x136]: Video: h264 (Main), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 50 tbr, 90k tbn, 50 tbc Stream #0:10x352: Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, fltp, 448 kb/s Stream #0:20x355: Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, fltp, 192 kb/s Stream #0:30x467: Subtitle: dvb_subtitle Stream #0:40x47e: Subtitle: dvb_subtitle Stream #0:50x13ec: Subtitle: dvb_teletext At least one output file must be specified

[Jpilk]

Once again I find that what I posted is not exactly what I see above. The new audio stream was reported as being (visual impaired) (descriptions), but while all the others show the language as (fin) that one has (dut)

:-)

[kmdewaal]

Yes, changes in the content such as in the audio and subtitle streams is what the mpegts-mythtv does handle and that the ffmpeg-mpegts does not. This is the main thing that needs to be merged with the latest ffmpeg-mpegts to get a new mpegts-mythtv. Another thing is that the mpegts-mytthv ignores the size of the PMT and always limits it to one packet. According to the documentation this is to fix a bug in BBC IPTV/iPlayer recordings who report the PMT size incorrect. I am hesitant to merge this as well because that code is really old, probably from the early days of IPTV, and is most likely not needed anymore. If there are people who actually have BBC IPTV/iPlayer recordings please try if they play correct with the ffmpeg-mpegts demuxer selected. And about the language (dut), I have tried all languages in that clip but I am certain it is not Dutch!

[jarnos]

And about the language (dut), I have tried all languages in that clip but I am certain it is not Dutch!

It has the Finnish translation of speech in the program mixed in. Dutch stream is used for it by Yle. It is made for people that do not see well. Maybe there is some way to recognize such an audio track. Reference: http://yle.fi/aihe/artikkeli/2014/12/04/tekstitys#aanitekstitys

Also Dutch subtitles are used for program subtitles when the broadcast language is Finnish of Swedish for Yle SD channels. User may want to choose that for the preferred subtitle language and some other for secondary subtitle language to get subtitles for all programs. For Yle HD channels there is a special subtitle track for people who do not hear well. Reference: https://yle.fi/aihe/artikkeli/2014/12/04/tekstitykset-ylen-kanavilla#miten-ohjelmatekstitys-nakyviin

[Jpilk]

Interesting, thanks. I had a picture of an expatriate community...

Klaas, re BBC iPlayer, I suspect that it is no longer available by the original route. paul-h may know more. get_player has usually worked for me, and seems by default to transcode to mp4

[kmdewaal]

The YLE recordings are completely according to the standards. What sets them apart is that they have very often a change in the number of streams; for example, a subtitle stream is added or removed. This happens usually at the start of a new program. In the MythTV demuxer (in file mpegts-mythtv.c, function pmt_cb) all streams are deleted and created again when this happens. This means that even when only e.g. a subtitle stream is added also the information of the video stream such as window size and frame rate is wiped. This is the root cause of the YLE playback problem; when the video stream information is missing the default window size and frame rate of 640x480 and 29.97Hz is used. Which shows both the age and the origins of MythTV... but that is another topic.

A relatively simple fix is, after a new PMT is received and after it is decided that the new PMT is different, to keep the streams that have not changed. As it happens the important streams, the video and the audio, are always at the beginning of the list of streams as retrieved from the PMT (program map table); the various subtitle streams come afterwards. Therefore stream comparison is done starting with stream 0 and counting upwards; the first stream that is different and all following streams with higher numbers are deleted and created again. This does fix the playback issues in the YLE stream and this will be committed to master after a few more tests.

About the FFmpeg demuxer. This demuxer does handle PMT changes correct but I have not figured out exactly where and when to implement the callbacks that are essential to MythTV reacting correct to stream-changes . Also, when using the FFmpeg demuxer, on some streams the subtitle language is not present while it is shown correct with the MythTV demuxer. In the slightly longer run a proper merge with the latest FFmpeg demuxer is definitely the plan but this should then also include fixes to the subtitle handling to avoid regressions.

[jarnos]

Apparently the fix is in this commit https://github.com/MythTV/mythtv/commit/ebeea33ad284f8827cd6caed6b30d100c4132eaf

I wonder in which version of MythTV (or mythtv-frontend package) the bug is fixed. The bug is still in 2:31.0+fixes.202105011948.b2ed400037~ubuntu20.04.1

[kmdewaal]

The bug is fixed in master. I have no confirmation if this really fixes the issue but there have also no regressions been reported. If there is a report that this really fixes the issue I can backport the fix to v31.

[jarnos]

Oh, I wish I could test it in Xubuntu 20.04.

[Jpilk]

I can confirm that current master plays, from the beginning and with no obvious problems, the file that I downloaded (see above) on 2 April. Filesize is 871.4 Mib. But I have no access to other recordings from this channel. HTH

[jarnos]

There is another extraction of a recording from Yle TV1: https://code.mythtv.org/trac/ticket/13557#comment:24

[Jpilk]

OK. I misunderstood at first, but now once again have that clip, named zoomed.ts, 16.2 MiB.

mythcommflag --rebuild again reports 'No I-frames found, rewinding...' and playback starts with 6 OK intro frames before going on, as hoped, to the railway station clip at frame 7 and onward. el7, software decode.

[jarnos]

I guess the fix is fine, if the video is not zoomed. (I could play the file by VLC and mpv without the zoom effect.)

[kmdewaal]

Thanks for the feedback. As the fix in master does appear to solve the YLE playback problem and there are no regressions seen with the playback of everything else the fix is now backported to fixes/31.