MythTV / mythtv

The official MythTV repository
https://www.mythtv.org
GNU General Public License v2.0
705 stars 345 forks source link

mythtranscode in MythTV v32 is half broken #581

Closed rolle5 closed 1 year ago

rolle5 commented 2 years ago

What steps will reproduce the bug?

I have upgraded my MythTV v31 installation to MythTV v32 with a complete reinstall. Now my customized mythtranscode jobs fail. The predefined mythtranscode profiles work fine (e.g. the transcoding profile: low profile). But my custom transcode job fails. I use the jobs to export parameters. These parameters go to a bash shell script. This has worked for many MythTV versions up to v31. Now in v32 it fails.

The command line in my shell script is: mythtranscode --chanid $CHANID --starttime $STARTTIMEUTC --honorcutlist --mpeg2 -o "video.cutted.mpg" -> in v32 fails (previously it worked)

Now I have halved the problem: The --mpeg2 option is the problematic option (for a lossless copy of the video stream). This option fails. Without this option it worked: mythtranscode --chanid $CHANID --starttime $STARTTIMEUTC --honorcutlist -o "video.cutted.mpg" -> works in v32

(Extra note: I made several attempts with the --mpeg2 option and with --honorcutlist and without --honorcutlist. Sometimes a video file with 0 bytes is created. Sometimes a video file is created with audio but without video.)

How often does it reproduce? Is there a required condition?

Always (I tried it several times with recordings from several TV channels and several recordings.)

What is the expected behaviour?

The option --mpeg2 (and --honorcutlist) should work as expected: to create a valid video file, that is cutted and a lossless transferation of the original video and audio streams.

What do you see instead?

A corrupted video file that has no stream marked as video.

Additional information

mythtranscode --version MythTV Version : v32.0+fixes.202206060103.7077a824d2~ubuntu22.04.1 MythTV Branch : fixes/32 Network Protocol : 91 Library API : 32.20200101-1 QT Version : 5.15.3 Options compiled in: linux profile use_hidesyms using_alsa using_oss using_pulse using_pulseoutput using_backend using_bindings_perl using_bindings_python using_bindings_php using_dvb using_firewire using_frontend using_hdhomerun using_satip using_vbox using_ceton using_joystick_menu using_libcec using_libcrypto using_gnutls using_libdns_sd using_libfftw3 using_libxml2 using_lirc using_mheg using_opengl using_egl using_qtwebkit using_qtscript using_qtdbus using_taglib using_v4l2 using_v4l2prime using_x11 using_system_libbluray using_system_libudfread using_systemd_notify using_systemd_journal using_drm using_bindings_perl using_bindings_python using_bindings_php using_freetype2 using_mythtranscode using_opengl using_egl using_drm using_vaapi using_nvdec using_vdpau using_ffmpeg_threads using_mheg using_libass using_libxml2 using_libmp3lame

rolle5 commented 2 years ago

For me it is the following situation: with both options: --honorcutlist --mpeg2 -> the generated video file has an audio channel and a corrupt video channel -> failed! with only one option: --mpeg2 -> the generated video file is 0 bytes in size -> failed! with only one option: --honorcutlist -> the generated video file is fine (audio channel + video channel) -> works!

The option --mpeg2 is faulty. Maybe there are some more regressions.

Jpilk commented 2 years ago

You don't say what format the input file is using, and you are asking for .mpg output, when .ts would be more usual nowadays.

I haven't used mythtranscode recently, because it would often produce files that mytharchive couldn't convert to DVD. Now I have several cutting scripts, for use with different input formats, that mostly work, but they assume a desktop environment.

What does 'mythffmpeg -i ' show for the video format of your recordings?

Or, during playback, M (for menu) > Playback > Playback Data > read Codec /Dec ?

rolle5 commented 2 years ago

You are right, the output name of my intermediate cutted file should end in .ts. I will change it in my script.

The format of the input file is a standard definition TV recording (Germany, DVB-C, PAL 720x576, MPEG2). This scenario has worked in recent years up to MythTV v31. I know that HD recordings (H264) do not work with mythtranscode (pity). (After that I usually convert the created intermediate cutted video file with ffmpeg and copy it in a seperate drawer.)

mythffmpeg -i 10014_20220609185100.ts ffmpeg version 4.4.1 Copyright (c) 2000-2021 the FFmpeg developers built with gcc 11 (Ubuntu 11.2.0-19ubuntu1) configuration: --arch=x86 --sysinclude=/usr/include --cc='ccache gcc' --cxx='ccache g++' --ld=gcc --cpu=generic --target_os=linux --pkg_config=pkg-config --prefix=/usr --libdir=/usr/lib --as=gcc --objcc=gcc --dep_cc=gcc --host_cc=gcc --host_ld=gcc --ar=ar --nm='nm -g' --ranlib='ranlib -D' --disable-cross-compile --enable-libmp3lame --enable-libx264 --disable-libx265 --enable-libvpx --disable-libxvid --enable-vdpau --enable-libxml2 --enable-libass --disable-dxva2 --enable-libbluray --disable-libfontconfig --disable-libfreetype --disable-libiec61883 --disable-crystalhd --enable-sdl2 --enable-ffplay --enable-vaapi --enable-libdrm --enable-gnutls --extra-cflags=' -w' --extra-ldflags= --extra-libs= --enable-stripping --strip='echo skipping strip' --disable-manpages --disable-podpages --disable-doc --disable-nvenc --enable-shared --disable-static --enable-gpl --enable-pic --disable-demuxer=mpegtsraw --disable-indev=dshow libavutil 56. 70.100 / 56. 70.100 libavcodec 58.134.100 / 58.134.100 libavformat 58. 76.100 / 58. 76.100 libavdevice 58. 13.100 / 58. 13.100 libavfilter 7.110.100 / 7.110.100 libswscale 5. 9.100 / 5. 9.100 libswresample 3. 9.100 / 3. 9.100 libpostproc 55. 9.100 / 55. 9.100 [mpegts @ 0x55d809a8e900] PES packet size mismatch [mpegts @ 0x55d809a8e900] Packet corrupt (stream = 2, dts = 3230969433). [mpegts @ 0x55d809a8e900] PES packet size mismatch [mpegts @ 0x55d809a8e900] Packet corrupt (stream = 3, dts = 3230975060). Input #0, mpegts, from '10014_20220609185100.ts': Duration: 00:05:24.12, start: 35575.947489, bitrate: 4574 kb/s Stream #0:00x20: Subtitle: dvb_teletext Stream #0:1[0xff]: Video: mpeg2video (Main), yuv420p(tv, top first), 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc Side data: cpb: bitrate max/min/avg: 15000000/0/0 buffer size: 1835008 vbv_delay: N/A Stream #0:20x100: Audio: mp2, 48000 Hz, stereo, fltp, 192 kb/s Stream #0:30x103: Audio: ac3, 48000 Hz, stereo, fltp, 384 kb/s


When the job is finished (with --honorcutlist --mpeg2) (mythtranscode --chanid $CHANID --starttime $STARTTIMEUTC --honorcutlist --mpeg2 -o "video.cutted.mpg"), the file looks like this:

mythffmpeg -i video.cutted.mpg ffmpeg version 4.4.1 Copyright (c) 2000-2021 the FFmpeg developers built with gcc 11 (Ubuntu 11.2.0-19ubuntu1) configuration: --arch=x86 --sysinclude=/usr/include --cc='ccache gcc' --cxx='ccache g++' --ld=gcc --cpu=generic --target_os=linux --pkg_config=pkg-config --prefix=/usr --libdir=/usr/lib --as=gcc --objcc=gcc --dep_cc=gcc --host_cc=gcc --host_ld=gcc --ar=ar --nm='nm -g' --ranlib='ranlib -D' --disable-cross-compile --enable-libmp3lame --enable-libx264 --disable-libx265 --enable-libvpx --disable-libxvid --enable-vdpau --enable-libxml2 --enable-libass --disable-dxva2 --enable-libbluray --disable-libfontconfig --disable-libfreetype --disable-libiec61883 --disable-crystalhd --enable-sdl2 --enable-ffplay --enable-vaapi --enable-libdrm --enable-gnutls --extra-cflags=' -w' --extra-ldflags= --extra-libs= --enable-stripping --strip='echo skipping strip' --disable-manpages --disable-podpages --disable-doc --disable-nvenc --enable-shared --disable-static --enable-gpl --enable-pic --disable-demuxer=mpegtsraw --disable-indev=dshow libavutil 56. 70.100 / 56. 70.100 libavcodec 58.134.100 / 58.134.100 libavformat 58. 76.100 / 58. 76.100 libavdevice 58. 13.100 / 58. 13.100 libavfilter 7.110.100 / 7.110.100 libswscale 5. 9.100 / 5. 9.100 libswresample 3. 9.100 / 3. 9.100 libpostproc 55. 9.100 / 55. 9.100 [mp3float @ 0x56436e46c600] Header missing Last message repeated 150 times [mp3float @ 0x56436e46c600] big_values too big [mp3float @ 0x56436e46c600] Error while decoding MPEG audio frame. [mp3float @ 0x56436e46c600] Header missing Last message repeated 1008 times [mpeg @ 0x56436e46a900] decoding for stream 0 failed [mpeg @ 0x56436e46a900] Could not find codec parameters for stream 0 (Audio: mp3 (mp3float), stereo, fltp, 96 kb/s): unspecified sample rate Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options Input #0, mpeg, from 'video.cutted.mpg': Duration: 00:02:48.01, start: 0.304711, bitrate: 4516 kb/s Stream #0:0[0x1e0]: Audio: mp3, stereo, fltp, 96 kb/s Stream #0:1[0x80]: Audio: ac3, 48000 Hz, stereo, fltp, 384 kb/s Stream #0:2[0x1c0]: Audio: mp2, 48000 Hz, stereo, s16p, 192 kb/s At least one output file must be specified


It seems that the stream 0 in the output file is wrong. I think stream 0 should be the video stream and not be the audio stream mp3.

Jpilk commented 2 years ago

It seems strange that mythffmpeg says stream #0 in the input file is dvb teletext. I have just looked at a few old recordings on a laptop running kubuntu 20.04 ppa master, and there stream #0 is video.

FWIW I'm still using the stream-selection logic from my old mythDVBcut script on the wiki.

Jpilk commented 2 years ago

I just noticed that mythffmpeg on my laptop was 'built with gcc 9 Ubuntu 9.4.0-1ubuntu1~20.04.1' while yours has gcc 11. ??

rolle5 commented 2 years ago

I just noticed that mythffmpeg on my laptop was 'built with gcc 9 Ubuntu 9.4.0-1ubuntu1~20.04.1' while yours has gcc 11. ??

I installed MythTV v32 about 2 weeks ago fresh from ppa:mythbuntu/32 on a fresh installation of Ubuntu 22.04LTS.

rolle5 commented 2 years ago

It seems strange that mythffmpeg says stream #0 in the input file is dvb teletext. I have just looked at a few old recordings on a laptop running kubuntu 20.04 ppa master, and there stream #0 is video.

FWIW I'm still using the stream-selection logic from my old mythDVBcut script on the wiki.

.ts is just a container. It can contain a wide variety of streams. In my opinion there are no restrictions concerning the type of streams, the number of streams and the order of the streams, so the position number 0 does not have to be occupied for video stream. I think it is a valid .ts file. Typical video programs (VLC, Totem, MythTVFrontend, ffplay/ffmpeg) have no problems with these video files.

Jpilk commented 2 years ago

Sorry, I was confused about the version you are running. The package version name has 20.04.1 but the platform is 22.04 LTS

And the dvb_teletext and video streams in the input are quoted as:

Stream #0:00x20: Subtitle: dvb_teletext Stream #0:1[0xff]: Video: mpeg2video (Main), yuv420p(tv, top first), 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc

which is probably what is confusing mythtranscode. ( Issue #402 mentions another interaction with teletext streams, comment of 3 Nov 2021...)

rolle5 commented 2 years ago

Ah, I must apologize, in the ticket header I was not 100% correct. I have now changed this. The rest is ok.

I now explain why in the ticket header was accidentally wrong version, why mythtranscode problem still exists and mythTV installation is clean, so there is no reason to confusing mythtranscode.

I have made 3 MythTV installations.

Nevertheless, all mythtranscode jobs and tests ran on the correct backend and all information came from the backend. I have now changed the correct backend mythtranscode version in the ticket header. Please reload this ticket web page again. -> so on the backend is a clean install and there can't be anything messed up, the problem with mythtranscode and the --mpeg2 option still exists. The information is reliable.

mythtranscode version (on my backend) is today after the last update v32.0+fixes.202206060103.7077a824d2~ubuntu22.04.1 The problem still exists.

rolle5 commented 2 years ago

Issue https://github.com/MythTV/mythtv/issues/402 mentions another interaction with teletext streams, comment of 3 Nov 2021.

This seems to be a completely different problem. My recordings (standard definition and HD) play back without any problems. They have been working with DVB and MythTV for about 10-15 years. And mythtranscode has also worked since then. I don't have a playback problem of the original recording, I have a problem with the output of the mythtranscode conversion.

My problem now is that about 2 weeks ago I did a complete reinstall with the new MythTV version and now I have the problems with the new mythtranscode version. My recordings (new ones made with MythTV v32 and older ones made with MythTV 31) work fine as always. I can also convert these files manually with ffmpeg. But mythtranscode (from MythTV v32) produces an incorrect output with the --mpeg2 option. When I try to play the produced output file with the players VLC, Totem, ffplay, there is only audio and no video. And also another conversion with ffmpeg is not possible. ffmpeg -i/mythffmpeg -i also shows that the output file contains incorrect stream information. It shows that there is no video stream in the generated file. Instead an additional mp3 audio stream has been added, which does not exist in the original file. The option --mpeg2 does no reencoding, but only a copy of existing streams (I always made my cuts only on keyframes). My current guess is that the --mpeg2 option is mishandling the video stream:

Jpilk commented 2 years ago

I tried it on a short recording from BBC1 with a cutlist, and confirm that '--honorcutlist --mpeg2' produced an audio-only file. mythffmpeg saw the original input stream #0:0 as video, 0:1 and 0:2 audio, 0:3 dvb_subtitle and 0:4, 0:5 unrecognised (interactive tv) streams.

Output file shows as Input #0, mpeg, from '11156_20220610125600.ts.tmp': Duration: 00:01:41.42, start: 0.286133, bitrate: 2932 kb/s Stream #0:0[0x1e0]: Audio: mp3, 24000 Hz, stereo, fltp, 8 kb/s Stream #0:1[0x1c0]: Audio: mp2, 48000 Hz, stereo, s16p, 256 kb/s

Without '--mpeg2' the output was nuv video + mp3

System is 'el7' at recent master (d0acf35cff, 5 June)

Jpilk commented 2 years ago

[john@HPFed ~]$ mythtranscode --chanid 20003 --starttime 20220611101900 --honorcutlist --mpeg2 --loglevel debug 2022-06-11 11:41:15.855728 C mythtranscode version: HEAD [v33-Pre-e377ae5632] www.mythtv.org 2022-06-11 11:41:15.855733 C Qt version: compile: 5.15.2, runtime: 5.15.2 2022-06-11 11:41:15.855758 I Fedora Linux 35 (Thirty Five) (x86_64) 2022-06-11 11:41:15.855762 N Enabled verbose msgs: general 2022-06-11 11:41:15.855767 N Setting Log Level to LOG_DEBUG 2022-06-11 11:41:15.866068 I Added logging to the console 2022-06-11 11:41:15.866420 I Setup Interrupt handler 2022-06-11 11:41:15.866428 I Setup Terminated handler 2022-06-11 11:41:15.866435 I Setup Segmentation fault handler 2022-06-11 11:41:15.866441 I Setup Aborted handler 2022-06-11 11:41:15.866447 I Setup Floating point exception handler 2022-06-11 11:41:15.866453 I Setup Illegal instruction handler 2022-06-11 11:41:15.866459 I Setup Bus error handler 2022-06-11 11:41:15.866474 I Setup Real-time signal 0 handler 2022-06-11 11:41:15.866480 I Setup Hangup handler 2022-06-11 11:41:15.866558 N Using runtime prefix = /usr 2022-06-11 11:41:15.866562 N Using configuration directory = /home/john/.mythtv 2022-06-11 11:41:15.866564 D appbindir = /usr/bin/ 2022-06-11 11:41:15.866567 D sharedir = /usr/share/mythtv/ 2022-06-11 11:41:15.866570 D libdir = /usr/lib64/mythtv/ 2022-06-11 11:41:15.866572 D themedir = /usr/share/mythtv/themes/ 2022-06-11 11:41:15.866575 D pluginsdir = /usr/lib64/mythtv/plugins/ 2022-06-11 11:41:15.866584 D translationsdir = /usr/share/mythtv/i18n/ 2022-06-11 11:41:15.866586 D filtersdir = /usr/lib64/mythtv/filters/ 2022-06-11 11:41:15.866589 D cachedir = /home/john/.mythtv/cache 2022-06-11 11:41:15.866592 D remotecachedir = /home/john/.mythtv/cache/remotecache 2022-06-11 11:41:15.866594 D themebasecachedir = /home/john/.mythtv/cache/themecache 2022-06-11 11:41:15.866597 D thumbnaildir = /home/john/.mythtv/cache/thumbnails 2022-06-11 11:41:15.866628 I Assumed character encoding: en_GB.UTF-8 2022-06-11 11:41:15.866833 I Using a profile name of: 'hp_fed1' (Usually the same as this host's name.) 2022-06-11 11:41:15.866847 I DefaultUPnP() - UPNP Search up to 2 secs 2022-06-11 11:41:15.867356 D MMulticastSocketDevice(:8): setsockopt - IP_MULTICAST_IF eno: Cannot assign requested address (99) 2022-06-11 11:41:15.968858 D MMulticastSocketDevice(:8): setsockopt - IP_MULTICAST_IF eno: Cannot assign requested address (99) 2022-06-11 11:41:16.227938 I DefaultUPnP() - UPNP Search up to 1 secs 2022-06-11 11:41:16.227987 D MMulticastSocketDevice(:8): setsockopt - IP_MULTICAST_IF eno: Cannot assign requested address (99) 2022-06-11 11:41:16.419270 D MMulticastSocketDevice(:8): setsockopt - IP_MULTICAST_IF eno: Cannot assign requested address (99) 2022-06-11 11:41:16.679148 I DefaultUPnP() - UPNP Search up to 1 secs 2022-06-11 11:41:16.679199 D MMulticastSocketDevice(:8): setsockopt - IP_MULTICAST_IF eno: Cannot assign requested address (99) 2022-06-11 11:41:16.912484 D MMulticastSocketDevice(:8): setsockopt - IP_MULTICAST_IF eno: Cannot assign requested address (99) 2022-06-11 11:41:17.995777 A Cannot find default UPnP backend 2022-06-11 11:41:17.995806 I Start up testing connections. DB 192.168.1.68, BE , attempt 0, status start, Delay: 2000 2022-06-11 11:41:17.995819 D PortChecker::checkPort(): host 192.168.1.68 port 3306 timeLimit 1000 linkLocalOnly 0 2022-06-11 11:41:18.496566 D PortChecker::checkPort(): socket state 3 2022-06-11 11:41:18.496776 D PortChecker::checkPort(): host 192.168.1.68 port 3306 timeLimit 1000 linkLocalOnly 0 2022-06-11 11:41:18.997432 D PortChecker::checkPort(): socket state 3 2022-06-11 11:41:19.001411 D PortChecker::checkPort(): host 192.168.1.68 port 3306 timeLimit 30000 linkLocalOnly 1 2022-06-11 11:41:19.005776 D FindDatabase() - Success! 2022-06-11 11:41:19.005848 D PortChecker::checkPort(): host 192.168.1.68 port 3306 timeLimit 30000 linkLocalOnly 1 2022-06-11 11:41:19.011043 N Setting QT default locale to en_GB 2022-06-11 11:41:19.011048 I Current locale en_GB 2022-06-11 11:41:19.011093 N Reading locale defaults from /usr/share/mythtv//locales/en_gb.xml 2022-06-11 11:41:19.048336 D PowerDBus: Acquired delay FD: 17 2022-06-11 11:41:19.144189 D PowerDBus: Max inhibit delay: 5seconds 2022-06-11 11:41:19.144411 I Power: On AC power 2022-06-11 11:41:19.144423 I Power: Supported actions: Suspend,Hibernate,Restart,Shutdown,HybridSleep 2022-06-11 11:41:19.145993 I Loading en_gb translation for module mythfrontend 2022-06-11 11:41:19.150790 D Setting 'Lossless transcoding' changed to 0 2022-06-11 11:41:19.150808 D Setting 'Resize video while transcoding' changed to 0 2022-06-11 11:41:19.150817 N Transcoding from /home/john/SGs/RecsSG4/20003_20220611101900.ts to /home/john/SGs/RecsSG4/20003_20220611101900.ts.tmp 2022-06-11 11:41:19.150823 I Honoring the cutlist while transcoding 2022-06-11 11:41:19.152512 I Opening /home/john/SGs/RecsSG4/20003_20220611101900.ts 2022-06-11 11:41:19.162384 I PES packet size mismatch 2022-06-11 11:41:19.162399 I Packet corrupt (stream = -155154144, dts = ▒). Handling Segmentation fault Segmentation fault (core dumped) [john@HPFed ~]$

Jpilk commented 2 years ago

And here's an analysis of the input file, renamed after (successful) cutting by my usual scripts.

[john@HPFed ~]$ mythffmpeg -i /home/john/SGs/RecsSG4/20003_20220611101900.ts_in76203.ts ffmpeg version 4.4.1 Copyright (c) 2000-2021 the FFmpeg developers built with gcc 11 (GCC) configuration: --arch=x86 --sysinclude=/usr/include --cc=gcc --cxx=g++ --ld=gcc --cpu=generic --target_os=linux --pkg_config=pkg-config --prefix=/usr --libdir=/usr/lib64 --as=gcc --objcc=gcc --dep_cc=gcc --host_cc=gcc --host_ld=gcc --ar=ar --nm='nm -g' --ranlib='ranlib -D' --disable-cross-compile --enable-libmp3lame --enable-libx264 --enable-libx265 --enable-libvpx --enable-libxvid --enable-vdpau --enable-libxml2 --enable-libass --disable-dxva2 --enable-libbluray --disable-libfontconfig --disable-libfreetype --disable-libiec61883 --disable-crystalhd --disable-sdl2 --disable-ffplay --enable-vaapi --enable-libdrm --enable-libaom --enable-libdav1d --enable-gnutls --extra-cflags=' -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-semantic-interposition -w' --extra-ldflags='-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 ' --extra-libs='-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 ' --enable-stripping --strip='echo skipping strip' --disable-manpages --disable-podpages --disable-doc --disable-nvenc --enable-shared --disable-static --enable-gpl --enable-pic --disable-demuxer=mpegtsraw --disable-indev=dshow libavutil 56. 70.100 / 56. 70.100 libavcodec 58.134.100 / 58.134.100 libavformat 58. 76.100 / 58. 76.100 libavdevice 58. 13.100 / 58. 13.100 libavfilter 7.110.100 / 7.110.100 libswscale 5. 9.100 / 5. 9.100 libswresample 3. 9.100 / 3. 9.100 libpostproc 55. 9.100 / 55. 9.100 [mpegts @ 0x5628a5479e00] PES packet size mismatch [mpegts @ 0x5628a5479e00] Packet corrupt (stream = 2, dts = 7798446156). Input #0, mpegts, from '/home/john/SGs/RecsSG4/20003_20220611101900.ts_in76203.ts': Duration: 00:03:01.08, start: 86469.041733, bitrate: 2311 kb/s Stream #0:0[0x6a5]: Video: mpeg2video (Main), yuv420p(tv, progressive), 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc Side data: cpb: bitrate max/min/avg: 15000000/0/0 buffer size: 1835008 vbv_delay: N/A Stream #0:10x6a6: Audio: mp2, 48000 Hz, stereo, fltp, 192 kb/s Stream #0:20x6a7: Audio: mp2, 48000 Hz, mono, fltp, 64 kb/s Stream #0:30x6c3: Subtitle: dvb_subtitle At least one output file must be specified [john@HPFed ~]$

Jpilk commented 2 years ago

It was my impression, from long ago, that mythtranscode problems often arose when DVB streams came or went. So I tried again, using as input the same recording but with only the video and main audio streams. I still get a segfault, this time without the complaint about PES packet size or corrupt stream.

Input #0, mpegts, from '/home/john/SGs/RecsSG4/20003_20220611101900.ts': Duration: 00:02:59.24, start: 1.400000, bitrate: 2229 kb/s Stream #0:0[0x100]: Video: mpeg2video (Main), yuv420p(tv, progressive), 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc Side data: cpb: bitrate max/min/avg: 3188800/0/0 buffer size: 1835008 vbv_delay: N/A Stream #0:1[0x101]: Audio: mp2, 48000 Hz, stereo, fltp, 192 kb/s At least one output file must be specified [john@HPFed ~]$

2022-06-11 13:17:01.693617 D Setting 'Resize video while transcoding' changed to 0 2022-06-11 13:17:01.693626 N Transcoding from /home/john/SGs/RecsSG4/20003_20220611101900.ts to /home/john/SGs/RecsSG4/20003_20220611101900.ts.tmp 2022-06-11 13:17:01.693633 I Honoring the cutlist while transcoding 2022-06-11 13:17:01.695299 I Opening /home/john/SGs/RecsSG4/20003_20220611101900.ts Handling Segmentation fault Segmentation fault (core dumped) [john@HPFed ~]$

Jpilk commented 2 years ago

Here are results for a similar sequence on the el7 system. Apologies for the strange formatting.

Input after stripping all but the main streams: [john@HP_Box RcSG2]$ mythffmpeg -i 11156_20220610125600.ts

Input #0, mpegts, from '11156_20220610125600.ts': Duration: 00:02:08.20, start: 1.400000, bitrate: 3072 kb/s Stream #0:0[0x100]: Video: mpeg2video (Main), yuv420p(tv, top first), 704x576 [SAR 16:11 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc Side data: cpb: bitrate max/min/avg: 5597200/0/0 buffer size: 1835008 vbv_delay: N/A Stream #0:1[0x101]: Audio: mp2, 48000 Hz, stereo, fltp, 256 kb/s At least one output file must be specified ===== [john@HP_Box ~]$ mythtranscode --chanid 11156 --starttime 20220610125600 --honorcutlist --mpeg2 --loglevel debug 2022-06-11 17:13:56.909256 C mythtranscode version: HEAD [v33-Pre-e2748d6e2b] www.mythtv.org 2022-06-11 17:13:56.909264 C Qt version: compile: 5.9.7, runtime: 5.9.7 2022-06-11 17:13:56.909293 I Scientific Linux 7.9 (Nitrogen) (x86_64) 2022-06-11 17:13:58.586424 D Setting 'Lossless transcoding' changed to 0 2022-06-11 17:13:58.586445 D Setting 'Resize video while transcoding' changed to 0 2022-06-11 17:13:58.586456 N Transcoding from /home/john/SGs/RcSG2/11156_20220610125600.ts to /home/john/SGs/RcSG2/11156_20220610125600.ts.tmp 2022-06-11 17:13:58.586465 I Honoring the cutlist while transcoding 2022-06-11 17:13:58.624330 I Opening /home/john/SGs/RcSG2/11156_20220610125600.ts 2022-06-11 17:13:58.641474 I ▒ #202852704, , mpegts '`�V': 2022-06-11 17:13:58.641511 I Duration: 140724806306144:140724806306144:02.08, start: ▒202852720.000001, bitrate: 140724806306160 kb/s 2022-06-11 17:13:58.641563 I Stream #202852288:202852288[0xc1747c0]: ▒▒0 ▒0 ▒0k ▒0 ▒ 2022-06-11 17:13:58.641568 I ▒Side data: 2022-06-11 17:13:58.641576 I ▒ cpb: bitrate max/min/avg: 202852272/202852272/0 buffer size: 0 vbv_delay: N/A 2022-06-11 17:13:58.641601 I Stream #202852288:202852288[0xc1747c0]: ▒ 2022-06-11 17:13:58.642197 I #0 PTS:00:00:01.440 Delta: 0.0ms queue: 1 2022-06-11 17:13:58.642211 I #1 PTS:00:00:01.424 Delta: 16ms queue: 2 2022-06-11 17:13:58.651984 I Del map found 0 at 168 (168) 2022-06-11 17:13:58.661876 I Mux rate: 5.95 Mbit/s 2022-06-11 17:13:58.981351 I Del map found 1 at 2703 (2703) 2022-06-11 17:13:59.393905 I Generating Keyframe Index 2022-06-11 17:13:59.393916 I Opening /home/john/SGs/RcSG2/11156_20220610125600.ts.tmp 2022-06-11 17:13:59.710692 I Header missing repeated many times 2022-06-11 17:13:59.712824 I Header missing 2022-06-11 17:13:59.712833 I Header missing 2022-06-11 17:13:59.712858 I Header missing 2022-06-11 17:13:59.712878 I invalid new backstep 202852080 Z�':-06-11 17:13:59.713960 I ▒ #202853360, , mpeg '�� 2022-06-11 17:13:59.713977 I Duration: 140724806306800:140724806306800:01.41, start: ▒202853376.000000, bitrate: 140724806306816 kb/s 2022-06-11 17:13:59.714017 I Stream #202852944:202852944[0xc174a50]: ▒ 2022-06-11 17:13:59.714039 I Stream #202852944:202852944[0xc174a50]: ▒ 2022-06-11 17:13:59.938473 N Transcode Completed 2022-06-11 17:13:59.982979 N Transcoding /home/john/SGs/RcSG2/11156_20220610125600.ts done 2022-06-11 17:13:59.983003 D PowerDBus: Releasing delay FD: 13 2022-06-11 17:13:59.983022 I PowerDBus: Closing interfaces 2022-06-11 17:13:59.984302 I Exiting ==== [john@HP_Box RcSG2]$ mythffmpeg -i 11156_20220610125600.ts.tmp libpostproc 55. 9.100 / 55. 9.100 [mp3float @ 0x5606e90b3900] Header missing Last message repeated 83 times [mp3float @ 0x5606e90b3900] invalid new backstep -1 Input #0, mpeg, from '11156_20220610125600.ts.tmp': Duration: 00:01:41.43, start: 0.376000, bitrate: 2932 kb/s Stream #0:0[0x1e0]: Audio: mp3, 24000 Hz, stereo, fltp, 8 kb/s Stream #0:1[0x1c0]: Audio: mp2, 48000 Hz, stereo, s16p, 256 kb/s At least one output file must be specified
Jpilk commented 2 years ago

The size of the .ts.tmp file above, and the reported bitrate, seemed high for audio-only. I haven't found a player that plays video from it, but mediainfo text view shows this:

General Complete name : /home/john/SGs/RcSG2/11156_20220610125600.ts.tmp Format : MPEG-PS File size : 35.5 MiB Duration : 1 min 41 s Overall bit rate mode : Variable Overall bit rate : 2 933 kb/s FileExtension_Invalid : mpeg mpg m2p vob vro pss evo

Video ID : 224 (0xE0) Format : MPEG Video Format version : Version 1 Format settings : BVOP Format settings, BVOP : Yes Format settings, Matrix : Default Format settings, GOP : M=3, N=24 Duration : 8 s 400 ms Bit rate mode : Variable Bit rate : 2 619 kb/s Width : pixel0 Height : pixel0 Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Scan type : Progressive Compression mode : Lossy Time code of first frame : 00:01:39:16 Time code source : Group of pictures header GOP, Open/Closed : Open Stream size : 2.62 MiB (7%)

Audio ID : 192 (0xC0) Format : MPEG Audio Format version : Version 1 Format profile : Layer 2 Duration : 1 min 41 s Bit rate mode : Constant Bit rate : 256 kb/s Channel(s) : 2 channels Sampling rate : 48.0 kHz Frame rate : 41.667 FPS (1152 SPF) Compression mode : Lossy Delay relative to video : 8 ms Stream size : 3.09 MiB (9%)

Jpilk commented 2 years ago

... and this was the input:

General ID : 1 (0x1) Complete name : /home/john/SGs/RcSG2/11156_20220610125600.ts Format : MPEG-TS File size : 47.0 MiB Duration : 2 min 8 s Overall bit rate mode : Variable Overall bit rate : 3 072 kb/s

Video ID : 256 (0x100) Menu ID : 1 (0x1) Format : MPEG Video Format version : Version 2 Format profile : Main@Main Format settings : CustomMatrix / BVOP Format settings, BVOP : Yes Format settings, Matrix : Custom Format settings, GOP : Variable Format settings, picture structure : Frame Codec ID : 2 Duration : 2 min 8 s Bit rate mode : Variable Bit rate : 2 662 kb/s Width : 704 pixels Height : 576 pixels Display aspect ratio : 16:9 Frame rate : 25.000 FPS Standard : PAL Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Scan type : Interlaced Scan order : Top Field First Compression mode : Lossy Bits/(Pixel*Frame) : 0.263 Time code of first frame : 00:00:00:00 Time code source : Group of pictures header GOP, Open/Closed : Open GOP, Open/Closed of first frame : Closed Stream size : 40.7 MiB (87%)

Audio ID : 257 (0x101) Menu ID : 1 (0x1) Format : MPEG Audio Format version : Version 1 Format profile : Layer 2 Codec ID : 3 Duration : 2 min 8 s Bit rate mode : Constant Bit rate : 256 kb/s Channel(s) : 2 channels Sampling rate : 48.0 kHz Frame rate : 41.667 FPS (1152 SPF) Compression mode : Lossy Delay relative to video : -40 ms Stream size : 3.91 MiB (8%)

Menu ID : 4096 (0x1000) Menu ID : 1 (0x1) Format : MPEG Video / MPEG Audio Duration : 2 min 8 s List : 256 (0x100) (MPEG Video) / 257 (0x101) (MPEG Audio) Service name : Service01 Service provider : FFmpeg Service type : digital television

Jpilk commented 2 years ago

I have tried to follow the wiki on debugging, using this gdbcommands file on a 2-stream postprocessed recording in F35

handle SIGPIPE nostop noprint handle SIG33 nostop noprint set logging enabled on set pagination off set breakpoint pending on break qFatal set args --chanid 20003 --starttime 20220611101900 --honorcutlist --mpeg2 --logpath /tmp -v general run thread apply all bt full set logging enabled off quit

The file at the logpath above stops at the segfault. Here's a section of the screen display. I'm not sure if the added 'quit' above stopped things prematurely, but ctrl-c just printed Quit, and didn't.

2022-06-13 12:13:02.820379 I Power: On AC power 2022-06-13 12:13:02.820393 I Power: Supported actions: Suspend,Hibernate,Restart,Shutdown,HybridSleep 2022-06-13 12:13:02.821971 I Loading en_gb translation for module mythfrontend 2022-06-13 12:13:02.826938 N Transcoding from /home/john/SGs/RecsSG4/20003_20220611101900.ts to /home/john/SGs/RecsSG4/20003_20220611101900.ts.tmp 2022-06-13 12:13:02.826945 I Honoring the cutlist while transcoding 2022-06-13 12:13:02.829034 I Opening /home/john/SGs/RecsSG4/20003_20220611101900.ts

Thread 1 "mythtranscode" received signal SIGSEGV, Segmentation fault. 0x00007ffff4a62a6a in __strlen_sse2 () from /usr/lib64/libc.so.6

Thread 7 (Thread 0x7fffd37fe640 (LWP 155535) "mythtranscode"):

0 0x00007ffff4acb94a in __futex_abstimed_wait_common () at /usr/lib64/libc.so.6

1 0x00007ffff4ace140 in pthread_cond_wait@@GLIBC_2.3.2 () at /usr/lib64/libc.so.6

2 0x00005555555850eb in MPEG2replex::Start() (this=0x5555557a8940) at mpeg2fix.cpp:571

cmccrae73 commented 2 years ago

MythTV version: v32.0+fixes.202205201849.b6286af7d8~ubuntu20.04.1

I can confirm that I have run into the same problem. I have a user job as follows:

mythtranscode --chanid %CHANID% --starttime %STARTTIMEUTC% --honorcutlist --mpeg2 -d

I am processing ATSC recordings (.ts).

The resulting file is the right size, and the audio is there but the video stream shows as follows in ffmpeg:

Stream #0:0[0x1e0]: Audio: mp3, 24000 Hz, stereo, fltp, 32 kb/s

Where a comparable unaffected recording reads as:

Stream #0:0[0x31]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], Closed Captions, 16999 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc

I don't know if there is a way to repair the damaged .ts

If there's anything further I can contribute, please let me know. Thanks.

Jpilk commented 2 years ago

I have no experience of mpeg2 video at comparable resolution and bitrate, but long ago had a report that mplex failed on such material; the solution then appeared to be to remux to .mkv. Has mythtranscode --mpeg2 ever worked for your files?

Jpilk commented 2 years ago

I get segfaults in Fedora35. 'el7' produces output with misidentified streams. gdb output below.

[john@HP_Box ~]$ cat ~/gdbcommands handle SIGPIPE nostop noprint handle SIG33 nostop noprint set logging on set pagination off set breakpoint pending on break qFatal set args --chanid 11156 --starttime 20220610125600 --honorcutlist --mpeg2 run thread apply all bt full set logging off quit

Run with: $ gdb mythtranscode -x ${HOME}/gdbcommands[john@HP_Box ~]$ [john@HP_Box ~]$ [john@HP_Box ~]$ [john@HP_Box ~]$ [john@HP_Box ~]$ [john@HP_Box ~]$ [john@HP_Box ~]$ gdb mythtranscode -x ${HOME}/gdbcommands GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-120.el7 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: ---Type to continue, or q to quit--- http://www.gnu.org/software/gdb/bugs/... Reading symbols from /usr/bin/mythtranscode...Reading symbols from /usr/lib/debug/usr/bin/mythtranscode.debug...done. done. Function "qFatal" not defined. Breakpoint 1 (qFatal) pending. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". [New Thread 0x7fffd53d6700 (LWP 18297)] [New Thread 0x7fffd4bd5700 (LWP 18298)] 2022-06-14 10:15:07.578320 C mythtranscode version: HEAD [v33-Pre-e2748d6e2b] www.mythtv.org 2022-06-14 10:15:07.578329 C Qt version: compile: 5.9.7, runtime: 5.9.7 2022-06-14 10:15:07.578365 I Scientific Linux 7.9 (Nitrogen) (x86_64) 2022-06-14 10:15:07.578369 N Enabled verbose msgs: general 2022-06-14 10:15:07.578376 N Setting Log Level to LOG_INFO 2022-06-14 10:15:07.607156 I Added logging to the console [New Thread 0x7fffcffff700 (LWP 18299)] 2022-06-14 10:15:07.608597 I Setup Interrupt handler 2022-06-14 10:15:07.608609 I Setup Terminated handler 2022-06-14 10:15:07.608619 I Setup Segmentation fault handler 2022-06-14 10:15:07.608632 I Setup Aborted handler 2022-06-14 10:15:07.608642 I Setup Floating point exception handler 2022-06-14 10:15:07.608652 I Setup Illegal instruction handler 2022-06-14 10:15:07.608661 I Setup Bus error handler 2022-06-14 10:15:07.608674 I Setup Real-time signal 0 handler 2022-06-14 10:15:07.608683 I Setup Hangup handler 2022-06-14 10:15:07.608833 N Using runtime prefix = /usr 2022-06-14 10:15:07.608837 N Using configuration directory = /home/john/.mythtv 2022-06-14 10:15:07.608897 I Assumed character encoding: en_GB.UTF-8 2022-06-14 10:15:07.609194 I Using a profile name of: 'hpbox' (Usually the same as this host's name.) 2022-06-14 10:15:07.609223 I Start up testing connections. DB localhost, BE , attempt 0, status start, Delay: 2000 [New Thread 0x7fffcf2fe700 (LWP 18302)] [Thread 0x7fffcf2fe700 (LWP 18302) exited] [New Thread 0x7fffcf2fe700 (LWP 18305)] 2022-06-14 10:15:09.148528 N Setting QT default locale to en_GB 2022-06-14 10:15:09.148535 I Current locale en_GB 2022-06-14 10:15:09.148731 N Reading locale defaults from /usr/share/mythtv//locales/en_gb.xml [New Thread 0x7fffce9d2700 (LWP 18314)] 2022-06-14 10:15:09.248053 I Power: Unknown power source 2022-06-14 10:15:09.248067 I Power: Supported actions: Suspend,Restart,Shutdown 2022-06-14 10:15:09.249143 I Loading en_gb translation for module mythfrontend 2022-06-14 10:15:09.253009 N Transcoding from /home/john/SGs/RcSG2/11156_20220610125600.ts to /home/john/SGs/RcSG2/11156_20220610125600.ts.tmp 2022-06-14 10:15:09.253020 I Honoring the cutlist while transcoding 2022-06-14 10:15:09.263921 I Opening /home/john/SGs/RcSG2/11156_20220610125600.ts 2022-06-14 10:15:09.298239 I ▒ #-11648, , mpegts ' �sUUU': 2022-06-14 10:15:09.298256 I Duration: 140737488343680:140737488343680:02.08, start: ▒-11632.000001, bitrate: 140737488343696 kb/s 2022-06-14 10:15:09.298304 I Stream #-12064:-12064[0xffffd0e0]: ▒▒0 ▒0 ▒0k ▒0 ▒ 2022-06-14 10:15:09.298309 I ▒Side data: 2022-06-14 10:15:09.298318 I ▒ cpb: bitrate max/min/avg: -12080/-12080/0 buffer size: 0 vbv_delay: N/A 2022-06-14 10:15:09.298341 I Stream #-12064:-12064[0xffffd0e0]: ▒ 2022-06-14 10:15:09.298900 I #0 PTS:00:00:01.440 Delta: 0.0ms queue: 1 2022-06-14 10:15:09.298912 I #1 PTS:00:00:01.424 Delta: 16ms queue: 2 2022-06-14 10:15:09.308209 I Del map found 0 at 168 (168) 2022-06-14 10:15:09.319310 I Mux rate: 5.95 Mbit/s 2022-06-14 10:15:09.644360 I Del map found 1 at 2703 (2703) [Thread 0x7fffce9d2700 (LWP 18314) exited] 2022-06-14 10:15:10.044936 I Generating Keyframe Index 2022-06-14 10:15:10.044947 I Opening /home/john/SGs/RcSG2/11156_20220610125600.ts.tmp 2022-06-14 10:15:10.365729 I Header missing 2022-06-14 10:15:10.365763 I Header missing 2022-06-14 10:15:10.365798 I Header missing 2022-06-14 10:15:10.365808 I Header missing 2022-06-14 10:15:10.365838 I Header missing 2022-06-14 10:15:10.365888 I Header missing 2022-06-14 10:15:10.365907 I Header missing 2022-06-14 10:15:10.365950 I Header missing 2022-06-14 10:15:10.366023 I Header missing 2022-06-14 10:15:10.366045 I Header missing 2022-06-14 10:15:10.366065 I Header missing 2022-06-14 10:15:10.366111 I Header missing 2022-06-14 10:15:10.366140 I Header missing 2022-06-14 10:15:10.366226 I Header missing 2022-06-14 10:15:10.366430 I Header missing 2022-06-14 10:15:10.366461 I Header missing 2022-06-14 10:15:10.366505 I Header missing 2022-06-14 10:15:10.366537 I Header missing 2022-06-14 10:15:10.366546 I Header missing 2022-06-14 10:15:10.366555 I Header missing 2022-06-14 10:15:10.366573 I Header missing 2022-06-14 10:15:10.366582 I Header missing 2022-06-14 10:15:10.366627 I Header missing 2022-06-14 10:15:10.366636 I Header missing 2022-06-14 10:15:10.366668 I Header missing 2022-06-14 10:15:10.366689 I Header missing 2022-06-14 10:15:10.366710 I Header missing 2022-06-14 10:15:10.366719 I Header missing 2022-06-14 10:15:10.366740 I Header missing 2022-06-14 10:15:10.366761 I Header missing 2022-06-14 10:15:10.366778 I Header missing 2022-06-14 10:15:10.366799 I Header missing 2022-06-14 10:15:10.366808 I Header missing 2022-06-14 10:15:10.366850 I Header missing 2022-06-14 10:15:10.366859 I Header missing 2022-06-14 10:15:10.366880 I Header missing 2022-06-14 10:15:10.366906 I Header missing 2022-06-14 10:15:10.366925 I Header missing 2022-06-14 10:15:10.366956 I Header missing 2022-06-14 10:15:10.367188 I Header missing 2022-06-14 10:15:10.367208 I Header missing 2022-06-14 10:15:10.367244 I Header missing 2022-06-14 10:15:10.367265 I Header missing 2022-06-14 10:15:10.367288 I Header missing 2022-06-14 10:15:10.367309 I Header missing 2022-06-14 10:15:10.367357 I Header missing 2022-06-14 10:15:10.367376 I Header missing 2022-06-14 10:15:10.367398 I Header missing 2022-06-14 10:15:10.367421 I Header missing 2022-06-14 10:15:10.367430 I Header missing 2022-06-14 10:15:10.367450 I Header missing 2022-06-14 10:15:10.367488 I Header missing 2022-06-14 10:15:10.367516 I Header missing 2022-06-14 10:15:10.367538 I Header missing 2022-06-14 10:15:10.367547 I Header missing 2022-06-14 10:15:10.367568 I Header missing 2022-06-14 10:15:10.367588 I Header missing 2022-06-14 10:15:10.367598 I Header missing 2022-06-14 10:15:10.367618 I Header missing 2022-06-14 10:15:10.367640 I Header missing 2022-06-14 10:15:10.367672 I Header missing 2022-06-14 10:15:10.367700 I Header missing 2022-06-14 10:15:10.367721 I Header missing 2022-06-14 10:15:10.367730 I Header missing 2022-06-14 10:15:10.367749 I Header missing 2022-06-14 10:15:10.367770 I Header missing 2022-06-14 10:15:10.367788 I Header missing 2022-06-14 10:15:10.367810 I Header missing 2022-06-14 10:15:10.367832 I Header missing 2022-06-14 10:15:10.367850 I Header missing 2022-06-14 10:15:10.367886 I Header missing 2022-06-14 10:15:10.367908 I Header missing 2022-06-14 10:15:10.367917 I Header missing 2022-06-14 10:15:10.367925 I Header missing 2022-06-14 10:15:10.367947 I Header missing 2022-06-14 10:15:10.367956 I Header missing 2022-06-14 10:15:10.367964 I Header missing 2022-06-14 10:15:10.367996 I Header missing 2022-06-14 10:15:10.368016 I Header missing 2022-06-14 10:15:10.368025 I Header missing 2022-06-14 10:15:10.368068 I Header missing 2022-06-14 10:15:10.368096 I Header missing 2022-06-14 10:15:10.368106 I Header missing 2022-06-14 10:15:10.368127 I Header missing 2022-06-14 10:15:10.368147 I invalid new backstep -12272 2022-06-14 10:15:10.369221 I ▒ #-10992, , mpeg '�����': 2022-06-14 10:15:10.369235 I Duration: 140737488344336:140737488344336:01.41, start: ▒-10976.000000, bitrate: 140737488344352 kb/s 2022-06-14 10:15:10.369272 I Stream #-11408:-11408[0xffffd370]: ▒ 2022-06-14 10:15:10.369294 I Stream #-11408:-11408[0xffffd370]: ▒ 2022-06-14 10:15:10.594310 N Transcode Completed 2022-06-14 10:15:10.596430 N Transcoding /home/john/SGs/RcSG2/11156_20220610125600.ts done 2022-06-14 10:15:10.596461 I PowerDBus: Closing interfaces 2022-06-14 10:15:10.597688 I Exiting [Thread 0x7fffd4bd5700 (LWP 18298) exited] [Thread 0x7fffd53d6700 (LWP 18297) exited] [Thread 0x7fffcffff700 (LWP 18299) exited] [Thread 0x7fffcf2fe700 (LWP 18305) exited] [Inferior 1 (process 18293) exited normally] [john@HP_Box ~]$

kmdewaal commented 2 years ago

While I largely miss the point of doing transcoding at all I do think that MythTV programs should not crash and this one is easily reproduced. I will have a look at it.

kmdewaal commented 2 years ago

The behavior reported by John on F35 can be reproduced with master on F36 with the following command: [klaas@kasus tmp]$ mythtranscode -i 71654_20220103192311.ts --mpeg2 -o o.ts The first failing part of the lot output is similar to John's log:

CoreContext mpeg2fix.cpp:802:InitAV  Opening 71654_20220103192311.ts
CoreContext mpeg2fix.cpp:81:my_av_print   #-1005422875, , mpegts '':
CoreContext mpeg2fix.cpp:81:my_av_print    Duration: 140723946708408:139662741108499:00.05, start: -1005422795.091679, bitrate: 140723946708424 kb/s
CoreContext mpeg2fix.cpp:81:my_av_print    Stream #-656745432:-1005423806[0xd8dadc28]: 0 0 0k 0 
CoreContext mpeg2fix.cpp:81:my_av_print  Side data:
CoreContext mpeg2fix.cpp:81:my_av_print    cpb: bitrate max/min/avg: -656745448/-1005422592/0 buffer size: 0 vbv_delay: N/A
CoreContext mpeg2fix.cpp:81:my_av_print    Stream #-656745432:-1005423806[0xd8dadc28](): 

The incorrect output is caused by commit c38564ea0773ee666c015da4d9112e4b38603b48 "Replace two calls to vsprintf with QString::asprintf." dd Apr 27, 2020, which handles the printing of log messages of ffmpeg in the MythTV logging system.

Reverting this commit in master and testing again gives:

CoreContext mpeg2fix.cpp:804:InitAV  Opening 71654_20220103192311.ts
CoreContext mpeg2fix.cpp:83:my_av_print  Input #0, mpegts, from '71654_20220103192311.ts':
CoreContext mpeg2fix.cpp:83:my_av_print    Duration: 00:00:05.28, start: 91679.883467, bitrate: 2888 kb/s
CoreContext mpeg2fix.cpp:83:my_av_print    Stream #0:0[0x6ff]: Video: mpeg2video (Main), yuv420p(tv, top first), 544x576 [SAR 32:17 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
CoreContext mpeg2fix.cpp:83:my_av_print      Side data:
CoreContext mpeg2fix.cpp:83:my_av_print        cpb: bitrate max/min/avg: 15000000/0/0 buffer size: 1835008 vbv_delay: N/A
CoreContext mpeg2fix.cpp:83:my_av_print    Stream #0:1[0x700](deu): Audio: mp2, 48000 Hz, stereo, fltp, 192 kb/s

This part is now identical to the output of mythtranscode with fixes/31 using the same recording. The transcoding itself, which looks to be OK with fixes/31, still does not work so this is not the only regression.

The plan is to put this fix in master and to look for other suspicious commits in mythtranscode.

Jpilk commented 2 years ago

My gdb output in f35, mythtv-33.Pre.434.gc4dcb881b1-100.fc35.x86_64.rpm HTH

2022-06-15 16:36:02.418946 N Transcoding from /home/john/SGs/RecsSG4/20002_20220613095200.ts to /home/john/SGs/RecsSG4/20002_20220613095200.ts.tmp 2022-06-15 16:36:02.418953 I Honoring the cutlist while transcoding 2022-06-15 16:36:02.436360 I Opening /home/john/SGs/RecsSG4/20002_20220613095200.ts 2022-06-15 16:36:03.066432 I Input #0, mpegts, from '/home/john/SGs/RecsSG4/20002_20220613095200.ts': 2022-06-15 16:36:03.066445 I Duration: 00:02:27.64, start: 1.400000, bitrate: 2242 kb/s 2022-06-15 16:36:03.066497 I Stream #0:0[0x100]: Video: mpeg2video (Main), yuv420p(tv, top first), 704x576 [SAR 16:11 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc 2022-06-15 16:36:03.066502 I Side data: 2022-06-15 16:36:03.066511 I cpb: bitrate max/min/avg: 4281600/0/0 buffer size: 1835008 vbv_delay: N/A 2022-06-15 16:36:03.066543 I Stream #0:1[0x101]: Audio: mp2, 48000 Hz, stereo, fltp, 256 kb/s 2022-06-15 16:36:03.067268 I #0 PTS:00:00:01.440 Delta: 0.0ms queue: 1 2022-06-15 16:36:03.067283 I #1 PTS:00:00:01.424 Delta: 16ms queue: 2 2022-06-15 16:36:03.079888 I Del map found 0 at 225 (225) 2022-06-15 16:36:03.098812 E avcodec_encode_video2 failed (-22) 2022-06-15 16:36:03.099110 E Transcoding /home/john/SGs/RecsSG4/20002_20220613095200.ts failed 2022-06-15 16:36:03.099154 I PowerDBus: Closing interfaces

linuxdude42 commented 2 years ago

I restored the commit that caused the logging problem, and fixed it to use the varargs version of the QString sprintf function. I have tested the output on both fixes-32 and master. This doesn't address the problem of the video track being misidentified.

KungFuJesus commented 2 years ago

I think this is a duplicate of this: https://github.com/MythTV/mythtv/issues/560

Jpilk commented 2 years ago

The log now shows well-formatted stream analyses, but (as expected) the output still isn't what was intended.

[john@HP_Box ~]$ mythtranscode --chanid 11156 --starttime 20220610125600 --honorcutlist --mpeg2 --loglevel debug 2022-06-16 10:03:05.329796 C mythtranscode version: HEAD [v33-Pre-2d28f6424e] www.mythtv.org 2022-06-16 10:03:05.329803 C Qt version: compile: 5.9.7, runtime: 5.9.7 2022-06-16 10:03:05.329832 I Scientific Linux 7.9 (Nitrogen) (x86_64)

2022-06-16 10:03:07.019714 I Opening /home/john/SGs/RcSG2/11156_20220610125600.ts 2022-06-16 10:03:07.025988 I Input #0, mpegts, from '/home/john/SGs/RcSG2/11156_20220610125600.ts': 2022-06-16 10:03:07.026003 I Duration: 00:02:08.20, start: 1.400000, bitrate: 3072 kb/s 2022-06-16 10:03:07.026056 I Stream #0:0[0x100]: Video: mpeg2video (Main), yuv420p(tv, top first), 704x576 [SAR 16:11 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc 2022-06-16 10:03:07.026061 I Side data: 2022-06-16 10:03:07.026069 I cpb: bitrate max/min/avg: 5597200/0/0 buffer size: 1835008 vbv_delay: N/A 2022-06-16 10:03:07.026093 I Stream #0:1[0x101]: Audio: mp2, 48000 Hz, stereo, fltp, 256 kb/s 2022-06-16 10:03:07.026617 I #0 PTS:00:00:01.440 Delta: 0.0ms queue: 1 2022-06-16 10:03:07.026628 I #1 PTS:00:00:01.424 Delta: 16ms queue: 2 2022-06-16 10:03:07.036489 I Del map found 0 at 168 (168) 2022-06-16 10:03:07.045952 I Mux rate: 5.95 Mbit/s 2022-06-16 10:03:07.182838 I Del map found 1 at 1262 (1262) 2022-06-16 10:03:07.420027 I Generating Keyframe Index 2022-06-16 10:03:07.420038 I Opening /home/john/SGs/RcSG2/11156_20220610125600.ts.tmp 2022-06-16 10:03:07.733796 I Header missing 2022-06-16 10:03:07.733831 I Header missing 2022-06-16 10:03:07.733859 I Header missing

... a page full of this

2022-06-16 10:03:07.735862 I Header missing 2022-06-16 10:03:07.735871 I Header missing 2022-06-16 10:03:07.735891 I Header missing 2022-06-16 10:03:07.735911 I invalid new backstep -1 2022-06-16 10:03:07.737082 I Input #0, mpeg, from '/home/john/SGs/RcSG2/11156_20220610125600.ts.tmp': 2022-06-16 10:03:07.737096 I Duration: 00:00:43.78, start: 0.376000, bitrate: 2927 kb/s 2022-06-16 10:03:07.737135 I Stream #0:0[0x1e0]: Audio: mp3, 24000 Hz, stereo, fltp, 8 kb/s 2022-06-16 10:03:07.737157 I Stream #0:1[0x1c0]: Audio: mp2, 48000 Hz, stereo, s16p, 256 kb/s 2022-06-16 10:03:07.835851 N Transcode Completed 2022-06-16 10:03:07.840952 N Transcoding /home/john/SGs/RcSG2/11156_20220610125600.ts done 2022-06-16 10:03:07.840972 D PowerDBus: Releasing delay FD: 13 2022-06-16 10:03:07.840987 I PowerDBus: Closing interfaces 2022-06-16 10:03:07.842045 I Exiting

As before, the output plays as an audio file.

Something that I hadn't seen before: After this operation the input file is unchanged. But it has lost its seektable, although not its cutpoints. Probably a result of 'Generating Keyframe Index'

linuxdude42 commented 2 years ago

I've tracked down the commit that caused the problem. I'm working through it to see which change in the commit caused the problem.

kmdewaal commented 2 years ago

I did not yet find the time to look further into this so great that your are looking into this David. My best guess is that commit 4b0ad53ca2e2c2494aa943a519705716e674d863 is causing at least one of the problems but I am not sure that this covers all. Happy hunting!

linuxdude42 commented 2 years ago

The fixes-32 version of mythtranscode should work now. The version of mythtranscode in master still fails, for a completely different reason. I'm opening a new issue for that.

linuxdude42 commented 2 years ago

The output of the myth-32 version of mythtranscode does have these two new lines that aren't in the mythtv-31 version:

2022-06-17 15:54:52.066606 I  stream 1 : no PTS found at end of file, duration not set
2022-06-17 15:54:52.066608 I  stream 2 : no PTS found at end of file, duration not set

These appeared with the upgrade of FFmpeg to version 4.3.1.

kmdewaal commented 2 years ago

The fixes/32 version of mythtranscode still fails here on F36, with the same errors as reported in #588. It looks indeed to be related to the FFmpeg update.

KungFuJesus commented 2 years ago

Is bare ffmpeg able to transcode the mpegts to plain mpeg2 (perhaps sans a cutlist?).

linuxdude42 commented 2 years ago

Are you sure you tested the right version of mythtranscode? I am also testing on on F36 using the latest fixes-32 mythtranscode, and I was able to transcode a dozen streams (sizes from 0.2G to 28G) and they all completed successfully. One of them threw 175 "packet corrupt" messages, but it completed.

The only time I see the "avcodec_encode_video2 failed" message is when I run mythtranscode from master.

kmdewaal commented 2 years ago

71654.ZIP

About versions, I do run everything on the same machine with only one database so there is room for mistakes. It is possible that some database setting does have an influence or that there is something going wrong with library paths. However, transcoding with fixes/31 goes OK on the same machine. It could also be depending on the input file and I've attached the file that I have used.

Please try it, just 4 seconds of a commercial. Here it transcodes correct on fixes/31 and it fails on the latest fixes/32 and master. Excerpt from the output on fixes/32:

2022-06-19 10:21:45.293862 I [553999/553999] CoreContext mpeg2fix.cpp:799:InitAV  Opening /home/klaas/m32-ts/tmp/71654_20220103192311.ts
2022-06-19 10:21:45.301427 I [553999/553999] CoreContext mpeg2fix.cpp:78:my_av_print  Input #0, mpegts, from '/home/klaas/m32-ts/tmp/71654_20220103192311.ts':
2022-06-19 10:21:45.301447 I [553999/553999] CoreContext mpeg2fix.cpp:78:my_av_print    Duration: 00:00:05.28, start: 91679.883467, bitrate: 2888 kb/s
2022-06-19 10:21:45.301520 I [553999/553999] CoreContext mpeg2fix.cpp:78:my_av_print    Stream #0:0[0x6ff]: Video: mpeg2video (Main), yuv420p(tv, top first), 544x576 [SAR 32:17 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
2022-06-19 10:21:45.301524 I [553999/553999] CoreContext mpeg2fix.cpp:78:my_av_print      Side data:
2022-06-19 10:21:45.301534 I [553999/553999] CoreContext mpeg2fix.cpp:78:my_av_print        cpb: bitrate max/min/avg: 15000000/0/0 buffer size: 1835008 vbv_delay: N/A
2022-06-19 10:21:45.301570 I [553999/553999] CoreContext mpeg2fix.cpp:78:my_av_print    Stream #0:1[0x700](deu): Audio: mp2, 48000 Hz, stereo, fltp, 192 kb/s
2022-06-19 10:21:45.304415 I [553999/553999] CoreContext mpeg2fix.cpp:2101:Start  #0 PTS:25:28:00.200 Delta: 0.0ms queue: 11
2022-06-19 10:21:45.304433 I [553999/553999] CoreContext mpeg2fix.cpp:2109:Start  #1 PTS:25:28:00.195 Delta: 5.51111ms queue: 2
2022-06-19 10:21:45.331147 E [553999/553999] CoreContext mpeg2fix.cpp:1293:BuildFrame  avcodec_encode_video2 failed (-22)
2022-06-19 10:21:45.331685 E [553999/553999] CoreContext main.cpp:746:main  Transcoding /home/klaas/m32-ts/tmp/71654_20220103192311.ts failed
2022-06-19 10:21:45.331713 I [553999/553999] CoreContext platforms/mythpowerdbus.cpp:72:~MythPowerDBus  PowerDBus: Closing interfaces
2022-06-19 10:21:45.332250 I [553999/553999] CoreContext mythcontext.cpp:1673:~MythContext  Exiting
rcrdnalor commented 2 years ago

On a pristine mythtv v32 installation (Ubuntu 22.04 via ppa fixes/32, updated today:) mythtranscode version: fixes/32 [v32.0+fixes.202206181511.a26786f5b4~ubuntu22.04.1] www.mythtv.org

the command mythtranscode --chanid <chanid> --starttime <starttime> --mpeg2 works and produces a file with video and audio streams, but mythtranscode --chanid <chanid> --starttime <starttime> --mpeg2 --honorcutlist fails with E avcodec_encode_video2 failed (-22) The cutlist is valid as seen by the logs:

2022-06-19 11:34:13.564038 I  Cutlist:
        0 - 6081
        59061 - 65733
        72370 - end
2022-06-19 11:34:14.187011 I  Del map found 0 at 6081 (6081)
2022-06-19 11:34:14.192441 E  avcodec_encode_video2 failed (-22)
linuxdude42 commented 2 years ago

My apologies. I was the one testing the wrong version of mythtranscode, still using one from my earlier bisect. I guess I really wanted the problem to be fixed. :-) When I use the latest fixes-32 (that contains FFmpeg 4.4.1) I see the avcodec_encode_video2 errors too. Like the master branch, this problem appears with the two commits https://github.com/MythTV/mythtv/commit/54f4c8f9b7ece971e5811a8b8a7ebb17d223874c / https://github.com/MythTV/mythtv/commit/4685f07ab81c2dd02f4acff4acf6443579dd4429.

Interestingly, if I use the --honorcutlist argument it fails much quicker.

rcrdnalor commented 2 years ago

Works for me on fixes/32: mythtranscode version: fixes/32 [v32.0+fixes.202206242147.501f85b389~ubuntu22.04.1] www.mythtv.org The command mythtranscode --chanid 10108 --starttime 20220420091600 --mpeg2 --honorcutlist -o ./Charmed.mpg succeeded and created an additional map file Charmed3.mpg.map without changing the seek table or the markup table of the original file.

Then, I added the transcodelossless flag to the codecparams of the MPEG2 recording profile and started transcoding within MythTV by M -> Job Options -> Begin Transcoding -> Default which created a lossless transcoded file recognized by MythTV with correctly updated seek and markup table. This file plays perfectly in the frontend, even on cut point transitions.

Albeit, the transcodelossless flag is not available within mythtv-setup anymore, it still exists in the code and is honored: https://github.com/MythTV/mythtv/blob/f86a7ee0dfb22b5a14fc8d8f2e3907f7d1bc8fda/mythtv/libs/libmythtv/recordingprofile.cpp#L1206

Thus, old databases may have this flag still set.

rolle5 commented 2 years ago

After a long time I tested it again. This time with version v32.0+fixes.202207042148.ced41a5f8f~ubuntu22.04.1 and now transcoding works for me again with both options "--honorcutlist" and "--mpeg2". Thanks a lot. This ticket can now be closed for me. I can't tell if it worked 1 or 2 versions earlier, as I haven't tested those. If there are any other bugs, you might want to open a separate ticket.