Difficulty in cutting h264 recordings

Platform:Fedora 35 x86_64
MythTV version:
Package version:mythtv-33.Pre.497.g8e36123478-100.fc35.x86_64.rpm
Component:

What steps will reproduce the bug?

How often does it reproduce? Is there a required condition?

What is the expected behaviour?

What do you see instead?

Additional information

I have returned to my scripts using 'ffmpeg' to cut h264 recordings. I have one that usually gives results acceptable to me, but only after a preliminary run in which no cuts are applied. Without that step the cutting is wildly inaccurate. It isn't clear to me exactly what makes the difference; I think it's probably to do with the timestamps.

FWIW the same script behaves similarly with mpeg2 recordings, but for those I know about, and use, alternatives.

Here are brief stream analyses for the final, intermediate and initial files:

Final, after cutting:

ionice -c 3 mythffmpeg -hide_banner -i /home/john/SGs/RecsSG2/20055_20220629205800.ts
[h264 @ 0x55b4ef7adf00] mmco: unref short failure Last message repeated 1 times Input #0, mpegts, from '/home/john/SGs/RecsSG2/20055_20220629205800.ts': Duration: 00:40:19.24, start: 1.400000, bitrate: 1172 kb/s Stream #0:0[0x100]: Video: h264 (High), yuv420p(tv, bt470bg, top first), 544x576 [SAR 64:33 DAR 544:297], 25 fps, 25 tbr, 90k tbn, 50 tbc Stream #0:10x101: Audio: aac_latm (LC), 48000 Hz, stereo, fltp At least one output file must be specified #############

Intermediate., uncut:

ionice -c 3 mythffmpeg -hide_banner -i /home/john/SGs/RecsSG2/20055_20220629205800.ts_370262in.ts [h264 @ 0x555f5c68cf00] mmco: unref short failure Last message repeated 1 times Input #0, mpegts, from '/home/john/SGs/RecsSG2/20055_20220629205800.ts_370262in.ts': Duration: 01:05:57.82, start: 1.400000, bitrate: 1179 kb/s Stream #0:0[0x100]: Video: h264 (High), yuv420p(tv, bt470bg, top first), 544x576 [SAR 64:33 DAR 544:297], 25 fps, 25 tbr, 90k tbn, 50 tbc Stream #0:10x101: Audio: aac_latm (LC), 48000 Hz, stereo, fltp At least one output file must be specified

##################

As recorded:

ionice -c 3 mythffmpeg -hide_banner -i /home/john/SGs/RecsSG2/20055_20220629205800.ts_369556in.ts [h264 @ 0x56079d5e4f00] non-existing PPS 0 referenced Last message repeated 1 times [h264 @ 0x56079d5e4f00] decode_slice_header error [h264 @ 0x56079d5e4f00] no frame! [h264 @ 0x56079d5e4f00] non-existing PPS 0 referenced Last message repeated 1 times

There's a full page of this, then

[h264 @ 0x56079d5e4f00] decode_slice_header error [h264 @ 0x56079d5e4f00] no frame! [h264 @ 0x56079d5e4f00] non-existing PPS 0 referenced Last message repeated 1 times [h264 @ 0x56079d5e4f00] decode_slice_header error [h264 @ 0x56079d5e4f00] no frame! [h264 @ 0x56079d5e4f00] mmco: unref short failure Last message repeated 1 times [mpegts @ 0x56079d5e0e00] start time for stream 2 is not set in estimate_timings_from_pts [mpegts @ 0x56079d5e0e00] stream 2 : no TS found at start of file, duration not set [mpegts @ 0x56079d5e0e00] Could not find codec parameters for stream 2 (Audio: aac_latm, 0 channels, fltp): unspecified sample rate Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options Input #0, mpegts, from '/home/john/SGs/RecsSG2/20055_20220629205800.ts_369556in.ts': Duration: 01:05:57.82, start: 59402.916156, bitrate: 1192 kb/s Stream #0:0[0x1d4d]: Video: h264 (High), yuv420p(tv, bt470bg, top first), 544x576 [SAR 64:33 DAR 544:297], 24.92 fps, 25 tbr, 90k tbn, 50 tbc Stream #0:10x1d4e: Audio: aac_latm (LC), 48000 Hz, stereo, fltp Stream #0:20x1d52: Audio: aac_latm, 0 channels, fltp Stream #0:30x1d51: Subtitle: dvb_subtitle At least one output file must be specified

###########

The problem occurs when 'ffmpeg' is cutting the wanted segments from the raw recording. They are much longer than intended.

[john@HPFed Bashtests]$ ls -Llh /home/john/SGs/RecsSG3/20055_20220702074100raw.ts -rw-r--r-- 1 john john 202M Jul 2 09:51 /home/john/SGs/RecsSG3/20055_20220702074100raw.ts [john@HPFed Bashtests]$ ls -lh /home/john/SGs/RecsSG3/20055_20220702074100cutfromraw.ts -rw-rw-r-- 1 john john 550M Jul 2 09:40 /home/john/SGs/RecsSG3/20055_20220702074100cutfromraw.ts

ionice -c3 mythffmpeg -hide_banner -ignore_unknown -noaccurate_seek -ss 4.9000 -t 54.8400 -i /home/john/SGs/RecsSG3/20055_20220702074100.ts -vcodec copy -acodec copy -avoid_negative_ts make_zero -f mpegts /var/RSG4/TScut_work/20055_20220702074100_184524_001.ts

Working from the raw recording: [john@HPFed Bashtests]$ ls -lh /var/RSG4/TScut_work/20055_20220702074100_182878_00.ts -rw-rw-r-- 1 john john 199M Jul 2 09:40 /var/RSG4/TScut_work/20055_20220702074100_182878_001.ts -rw-rw-r-- 1 john john 185M Jul 2 09:40 /var/RSG4/TScut_work/20055_20220702074100_182878_002.ts -rw-rw-r-- 1 john john 167M Jul 2 09:40 /var/RSG4/TScut_work/20055_20220702074100_182878_003.ts [john@HPFed Bashtests]$ [john@HPFed Bashtests]$ After preliminary run with no cutlist, mythcommflag--rebuild, reset the same cutlist... [john@HPFed Bashtests]$ ls -lh /var/RSG4/TScut_work/20055_20220702074100_184524_00.ts -rw-rw-r-- 1 john john 9.4M Jul 2 10:00 /var/RSG4/TScut_work/20055_20220702074100_184524_001.ts -rw-rw-r-- 1 john john 9.3M Jul 2 10:00 /var/RSG4/TScut_work/20055_20220702074100_184524_002.ts -rw-rw-r-- 1 john john 30M Jul 2 10:00 /var/RSG4/TScut_work/20055_20220702074100_184524_003.ts [john@HPFed Bashtests]$

######### ----Segments are

file /var/RSG4/TScut_work/20055_20220702074100_184524_001.ts file /var/RSG4/TScut_work/20055_20220702074100_184524_002.ts file /var/RSG4/TScut_work/20055_20220702074100_184524_003.ts

File segments have been written, Concatenating.

ionice -c3 mythffmpeg -hide_banner -ignore_unknown -fflags +genpts -f concat -safe 0 -i seglist_184524 -c copy /mnt/sdb1/Vids/20055_20220702074100_184524.ts Output file is /home/john/SGs/RecsSG3/20055_20220702074100.ts

I think the essential difference from the raw recording format is ' fflags +genpts '

Some thoughts came into my mind when reading this issue: Did you mean: "I've downloaded a program or recorded an aired DVB program and FFMPEG provided by MythTV master can not cope with this recording in respect to cut and transcode this recording?"

In order to eliminate any impacts which might not be under control of MythTV, please consider the following items:

Output data rely on the quality of the input. In order to check the input quality, you can use a script like https://forum.mythtv.org/viewtopic.php?f=2&t=4013&p=19636&hilit=check+recording+quality#p19688 or, if recorded via DVB, check for reception, e.g.: by https://www.linuxtv.org/wiki/index.php/Dvbv5-zap https://www.linuxtv.org/wiki/index.php/Dvb-fe-tool
In case of a DVB recording, you can try to eliminate MythTV completely, like saving a DVB stream to disk. See programs in https://www.linuxtv.org/wiki/index.php/TV_Related_Software
If you think that FFMPEG is part of your issue, then try static builds of FFMPEG, like https://johnvansickle.com/ffmpeg/
I successfully ran 'avidemux 2.7' on recordings in order to cut and trancode them. Please download and install from trusted source this version and report the results.
The issue mentions an output of 'ffmpeg -i' on a pristine recording:
```
As recorded:
ionice -c 3 mythffmpeg -hide_banner -i /home/john/SGs/RecsSG2/20055_20220629205800.ts_369556in.ts
```
Please confirm that this recording named with a non MythTV conpliant name is actually the recording 'as recorded by MythTV'.

Additional tweaks within MythTV which are worth to try:

MythTV provides the ability to remove unwanted streams during recording: What does mythtv-setup tell you in Recording Profiles -> Hardware DVB Encoders -> Default -> Recording Type If set to 'TV only', try 'all' instead.
Try 'mythtranscode' with the '--fifodir' option and compare results. See https://www.mythtv.org/wiki/Example_Script_using_mythtranscode_in_fifodir_mode https://www.mythtv.org/wiki/Mythtranscode

In general, this issue does not provide detailed information for me about the reason and reproducibility (is it a regression?). Please provide your scripts and the source file, i.e.: the pristine recording file.

Thank you for this very full response. I don't expect to try using many of the tools that you suggest soon, because my script now usually does what I want, but it's good to have the references.

I created this Issue because it appears that the underlying cause has existed for years. In particular, my script logs for cutting h264 recordings often complain, as below. I had found no useful suggestions about how to "Fix my code". I'm not suggesting that there has been a reversion.

"Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly"

You asked about non-MythTV-compliant filenames. I rename input files so that the script output can replace the original.

Thanks again.

Another reference that might be relevant here:

https://forum.mythtv.org/viewtopic.php?t=4400#p21724

Almost all broadcast mpeg-ts has an offset in AV packets (with same PTS) & never uses IDR I frames.. AIUI Mythtv uses the first video packet as time zero, almost all other AV software (mkvtoolnix, vlc etc) uses the first (earliest PTS). I've struggled to find a ffmpeg/ffprobe/mediainfo cmd to extract that offset.

A new aspect of this has come to light after a recent thread on the users list.

Using a command line like

mythffmpeg -fflags +genpts -i inputfile -acodec copy -vcodec copy -scodec copy -f mpegts outputfile

apparently allowed a suggested cutting script to work when the 'outfile' had replaced the original recording. This has been reported earlier in several other contexts.

It seemed worth trying the same procedure on the output of my old 'pycut.py' script, which simply copies and concatenates segments of the original recording, where the cuts are made at video keyframes. Direct playback had been mostly good, but usually with transients at the joins, where the timestamps jump. More recently I have used ffmpeg concatenation as outlined above; better but not perfect.

So far my experience with this new scheme is limited, but the transients seem much less noticeable during playback with both mythfrontend and leanfront. As I have it at present both the 'pycutting' and the remuxing are followed by seektable rebuilds and other actions needed to create a replacement recording, but it ought to be possible to drop the first set, and possibly to use pipes instead of multiple large working files.

My old scripts, in various states of revision, are still accessible through https://www.mythtv.org/wiki/MythDVBcut

MythTV / mythtv