Missing frames desync streams

[antoniu200]

Hi,

I have multiple TV .ts recordings I would like to burn to BDs to keep. TV being TV, the Electronic Program Guide is not 100% precise, so I set 5-10 minute extra recording margins, depending on the channel recorded, in which case most recordings also need to be cut before I save them. I use AviDemux to cut them at keyframes, so without re-encoding. Usually AviDemux muxes them in .mkv.

After AviDemux cuts them, I need to re-mux the subtitles, if available in DVBSub format, subtitles which I re-encode in PGS format to avoid OCR. For this purpose, I use tsMuxeR to remux the file as M2TS. Long story short, after muxing with tsMuxeR, today I have occurred a file that, after tsMuxeR muxes it, the video & audio fall out of sync slowly. I have tried both the CLI and GUI, but the results are identical. I have to mention that this file should not be much different than the other ones I have muxed with tsMuxeR, since nothing in the process above has been changed.

I have also tried using Cu Selur's Hybrid, which separates the audio & video using before muxing them together using tsMuxeR, but the result is identical.

MediaInfo on the cut MKV file:

General
Unique ID                                : 135821409883765255442555416544272178488 (0x662E408596BCCA9708836E9FD8BFDD38)
Format                                   : Matroska
Format version                           : Version 4
File size                                : 5.98 GiB
Duration                                 : 1 h 40 min
Overall bit rate                         : 8 555 kb/s
Writing application                      : Lavf58.76.100
Writing library                          : Lavf58.76.100
ErrorDetectionType                       : Per level 1

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 4 frames
Format settings, GOP                     : M=4, N=24
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 1 h 40 min
Bit rate                                 : 7 489 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 25.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Interlaced
Scan type, store method                  : Separated fields
Scan order                               : Top Field First
Bits/(Pixel*Frame)                       : 0.144
Stream size                              : 5.24 GiB (88%)
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Audio #1
ID                                       : 2
Format                                   : MPEG Audio
Format version                           : Version 1
Format profile                           : Layer 2
Codec ID                                 : A_MPEG/L2
Codec ID/Hint                            : MP2
Duration                                 : 1 h 40 min
Bit rate mode                            : Constant
Bit rate                                 : 256 kb/s
Channel(s)                               : 2 channels
Sampling rate                            : 48.0 kHz
Compression mode                         : Lossy
Delay relative to video                  : 134 ms
Stream size                              : 183 MiB (3%)
Language                                 : Romanian
Default                                  : Yes
Forced                                   : No

Audio #2
ID                                       : 3
Format                                   : AC-3
Format/Info                              : Audio Coding 3
Commercial name                          : Dolby Digital
Codec ID                                 : A_AC3
Duration                                 : 1 h 40 min
Bit rate mode                            : Constant
Bit rate                                 : 640 kb/s
Channel(s)                               : 6 channels
Channel layout                           : L R C LFE Ls Rs
Sampling rate                            : 48.0 kHz
Frame rate                               : 31.250 FPS (1536 SPF)
Compression mode                         : Lossy
Delay relative to video                  : 200 ms
Stream size                              : 458 MiB (7%)
Language                                 : qaa
Service kind                             : Complete Main
Default                                  : No
Forced                                   : No

Worth mentioning that tsMuxeR is instructed to disregard the MP2 audio source, to keep M2TS compatibility. Also not sure whether or not I should and how I would share the file, since it contains copyrighted content (movie) and the sync issues are only visible after an hour of playback.

Please advise. Thank you!

Meta file for instructions

[jcdr428]

@antoniu200 I can have a look: please PM me a link to download the file on doom9, user a5180007.

[antoniu200]

I have tried multiple things since and it seems like trimming with FFmpeg eliminated the issue, but it is not a reliable solution for the future. I will raise this issue with the devs over at AviDemux, the issue has something to do with "Non-monotonous DTS" caused by editlists.

I'll report back with news as soon as I get a reply from them.

EDIT: What is strange is the fact that I have cut before multiple .ts files from TV, but none of them exhibited such behavior after muxing with tsMuxeR before this one. I hope I'll be able to reproduce this issue with another file soon and send you a sample.

[antoniu200]

I encountered the bug again, on a raw .ts file from TV, not muxed or cut with any 3rd party program. It seems it has to do with certain types of bad frames, while tsMuxeR tries to resync the stream.

I have also noticed that, while muxing with tsMuxeR, timestamps are not precisely kept. For example, with subtitles, if they are straight up muxed using tsMuxeR without any kind of delay, they will not show up at the correct time, as they did in the original video file. Another thing is that the track delay is not kept as in the source, but rather tsMuxeR changes it automatically (and most of the times incorrectly), often leading to poorly synced audio.

I have just made an account over at doom9 so I can send you the sample, but I have to wait 5 days before I post or write messages. I hope Drive won't mark the video as copyright-infringing content by then.

EDIT: Just tried to mux it with FFmpeg (-c copy). The audio and video remained in sync until the end of the stream, although it ignored the MP2 audio.

[antoniu200]

@jcdr428 Hi! Just wanted to let you know I sent you the video that presents issues when remuxing a couple of days ago on doom9. Please let me know what your status on this issue is, when possible. No rush.

[jcdr428]

@antoniu200 I have looked at your sample: the problem is not really the bad frames, it is rather the missing frames -e.g. at 30min56sec, there is a whopping gap of 60 AC3 frames i.e. nearly 2 seconds.

tsMuxer would need to resync on the PTS of the source container, which is not the way tsMuxer currently works: tsMuxer looks at the elemental streams only. When an error is detected, tsMuxer simply increases by one frame duration. I am afraid that correcting this would be complicated and the issue is likely to be opened for a long time...

[antoniu200]

I have looked at your sample: the problem is not really the bad frames, it is rather the missing frames -e.g. at 30min56sec, there is a whopping gap of 60 AC3 frames i.e. nearly 2 seconds.

I'll edit the title accordingly. Well, that's exactly why I chose this video to send to you and not a different one - the issue should be most obvious in this one because of the big gaps.

tsMuxer would need to resync on the PTS of the source container, which is not the way tsMuxer currently works: tsMuxer looks at the elemental streams only. When an error is detected, tsMuxer simply increases by one frame duration. I am afraid that correcting this would be complicated and the issue is likely to be opened for a long time...

I have no problem keeping it open for a while, as long as the issue gets solved. Dropping it won't help anybody. Thing is, if I try to mux to MKV using MKV muxer, FFmpeg or AviDemux, the issue is not there. Do all of these programs resync when such issue appears? Is there any way I could tweak the elemental stream that's causing the issue while preserving the original duration and only then import it in tsMuxer?

[jcdr428]

@antoniu200 ffmpeg, mkvtoolnix etc. most likely use the container PTS -which is the right thing to do. Unfortunately there is no temp "tweaking" until tsMuxer is modified.