Help with MPEG TS Container support

[SecureTVBob]

Hi LiTr is really cool code as was your approach to transcoding.
We have an MPEG2 video in an MPEG TS container from an ATSC video tuner. LiTr properly transcodes the MPEG2 video to H.264 when the input video is an MP4 container. On some Set Top Box devices LiTr will not transcode the same MPEG2 video in an MPEG TS container. We adjusted LiTr code to accept mime/mpeg2 video from the file with Transport Stream container.

It seems like when the MPEG TS container with MPEG2 video is open some parameter is not set for the encoder. The Amlogic OMX encoder error is 0x8001001 which is an error for OMX to report "Unknown error."

Any suggestions on how to get the parameters configured properly so the encoder will work. What also is interesting is LiTr can decode the MPEG2 video in a MPEG TS container because extract frame and video filters can decode and play the video.

Any suggestions on how to debug or what settings encoder might need or how to fix?

This is link for MPEG2 video in MP4 container that works well with LiTr transcoder: https://drive.google.com/file/d/1VFoAL1xJNw6pAq-fp1rMhwMfwaQ3uP1o/view?usp=sharing

This link is for same MPEG2 video in MPEG TS container: https://drive.google.com/file/d/15Qmh12BFCJudwmgiTe8ucI5wdYfsWm24/view?usp=sharing

[izzytwosheds]

Thank you for enjoying LiTr! I didn't realize one could use LiTr on a set top box. Can you tell a bit more about how you guys use it? I am extremely curious.

I will look into this. I tried playing those two videos on my Android device (Samsung Galaxy Note 10), and they are behaving very strange. MP4 video does not play at all, TS video plays audio only. Tried with both built-in player and ExoPlayer, same problem on both. I think, something funky is going on with container parsing,MediaMetadataRetriever was not even able to extract a thumbnail. Can you guys share what you did to make this work, or is proprietary know-how?

As for encoder, one possible idea comes to mind is to maybe try to play around with different colors formats for the encoder. I wonder if decoder is producing frames in some format that is not compatible with what encoder expects. Shouldn't be the case, but might be worth a try. I will keep thinking about it.

[SecureTVBob]

Hi it is nice to hear from you.

On the STB some run AOSP and some run Android TV and we have one with AOSP. The Android STB chipsets have h/w accelerators using at the lower levels OMX drivers to the chip h/w encoder/decoder. Some tablets play both files without any problems and I believe there are few reasons for this related to the tablet chipset needing camera input and camera encoding along the lines of why you developed LiTr. The chipset vendors for the tablets may test their drivers better than the STB chipset vendors. STB chips not being used in cameras do not need the camera to encoder path and the driver software for raw camera input is not well tested or is not tested at all. In the case of the STBs we were testing on the older Amlogic chip S905X did not handle the MPEG TS container with MPEG2 Video but it did transcode the same MPEG 2 Video in MP4 container. My thought on this is that some settings for the encoder instance and more specifically lower level OMX driver settings pass through from Mediacodec was not setup because in the MIME parsing in LiTr support for MPEG TS container was not required.

The transcode process when MPEG2 video in MPEG TS container failed at the OMX initialize encoder call. The MPEG2 video in MP4 container worked perfect and looked good and transcode speed of 1920x1080 to 1280x720 was 2x realtime.

So to me it seems like some missing encoder initialization value because no recognized mime type in the setup. Do you think this could be the case and if so any ideas?

Regards,

Bob Kulakowski

CEO

Secure TV, LLC

Mobile: +1-760-533-5888

Main: +1-844-401-5777

Web: http://www.SecureTV.TV www.SecureTV.TV

Email: @.***

Address:

12707 High Bluff Drive

Suite 200

San Diego, CA 92130 USA

From: Izzat Bahadirov @. Sent: Tuesday, April 19, 2022 7:04 AM To: linkedin/LiTr @.> Cc: SecureTVBob @.>; Author @.> Subject: Re: [linkedin/LiTr] Help with MPEG TS Container support (Issue #187)

Thank you for enjoying LiTr! I didn't realize one could use LiTr on a set top box. Can you tell a bit more about how you guys use it? I am extremely curious.

I will look into this. I tried playing those two videos on my Android device (Samsung Galaxy Note 10), and they are behaving very strange. MP4 video does not play at all, TS video plays audio only. Tried with both built-in player and ExoPlayer, same problem on both. I think, something funky is going on with container parsing,MediaMetadataRetriever was not even able to extract a thumbnail. Can you guys share what you did to make this work, or is proprietary know-how?

As for encoder, one possible idea comes to mind is to maybe try to play around with different colors formats for the encoder. I wonder if decoder is producing frames in some format that is not compatible with what encoder expects. Shouldn't be the case, but might be worth a try. I will keep thinking about it.

— Reply to this email directly, view it on GitHub https://github.com/linkedin/LiTr/issues/187#issuecomment-1102697430 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AKQXAPSIAWNAVYBZUL26IMDVF24NHANCNFSM5TWVAJFA . You are receiving this because you authored the thread. https://github.com/notifications/beacon/AKQXAPUPZAP3WU2HRD7X7ILVF24NHA5CNFSM5TWVAJFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOIG45HVQ.gif Message ID: @. @.> >

[izzytwosheds]

I ran ffprobe, got some strange errors about invalid frame size and some corrupt packets:

[mpeg2video @ 0x7fb74d808800] Invalid frame dimensions 0x0. Last message repeated 30 times [mpegts @ 0x7fb74d00f400] PES packet size mismatch [mpegts @ 0x7fb74d00f400] Packet corrupt (stream = 1, dts = 359012687). [mpegts @ 0x7fb74d00f400] PES packet size mismatch [mpegts @ 0x7fb74d00f400] Packet corrupt (stream = 2, dts = 359012687). Input #0, mpegts, from 'kgtv-10-abc-atsc-capture.ts': Duration: 00:01:19.55, start: 3909.989856, bitrate: 8355 kb/s Program 3 Stream #0:0[0x31]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, top first), 1920x1080 [SAR 1:1 DAR 16:9], Closed Captions, 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc Side data: cpb: bitrate max/min/avg: 17598000/0/0 buffer size: 7995392 vbv_delay: N/A Stream #0:10x34: Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 384 kb/s Stream #0:20x35: Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, fltp, 192 kb/s

I wonder if this could be throwing off OMX codecs. QuickTime on Mac was able to play the .ts file, but with artifacts in the beginning.

[SecureTVBob]

I don’t think they would throw off the encoder setup. We see these type of messages until the first i-frame, ffmpeg is a bit noisy, and then even after timestamp errors are very common. In your code when you read the mime type and set parameters for the H.264 encoder do you know which ones must be set. I think it is this because when the mime type is parsed the mpeg ts container mime is not in the code and so the encoder may be missing some important parameters beyond bit rate, frame rate. Could it something like pixel format, or something related to surface details, etc. This is above what I know about encoders.

Regards,

Bob Kulakowski

CEO

Secure TV, LLC

Mobile: +1-760-533-5888

Main: +1-844-401-5777

Web: http://www.SecureTV.TV www.SecureTV.TV

Email: @.***

Address:

12707 High Bluff Drive

Suite 200

San Diego, CA 92130 USA

From: Izzat Bahadirov @. Sent: Wednesday, April 20, 2022 7:06 AM To: linkedin/LiTr @.> Cc: SecureTVBob @.>; Author @.> Subject: Re: [linkedin/LiTr] Help with MPEG TS Container support (Issue #187)

I ran ffprobe, got some strange errors about invalid frame size and some corrupt packets:

[mpeg2video @ 0x7fb74d808800] Invalid frame dimensions 0x0. Last message repeated 30 times [mpegts @ 0x7fb74d00f400] PES packet size mismatch [mpegts @ 0x7fb74d00f400] Packet corrupt (stream = 1, dts = 359012687). [mpegts @ 0x7fb74d00f400] PES packet size mismatch [mpegts @ 0x7fb74d00f400] Packet corrupt (stream = 2, dts = 359012687). Input #0, mpegts, from 'kgtv-10-abc-atsc-capture.ts': Duration: 00:01:19.55, start: 3909.989856, bitrate: 8355 kb/s Program 3 Stream #0:0[0x31]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, top first), 1920x1080 [SAR 1:1 DAR 16:9], Closed Captions, 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc Side data: cpb: bitrate max/min/avg: 17598000/0/0 buffer size: 7995392 vbv_delay: N/A Stream #0:10x34: Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 384 kb/s Stream #0:20x35: Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, fltp, 192 kb/s

I wonder if this could be throwing off OMX codecs. QuickTime on Mac was able to play the .ts file, but with artifacts in the beginning.

— Reply to this email directly, view it on GitHub https://github.com/linkedin/LiTr/issues/187#issuecomment-1103975574 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AKQXAPXNTFSG6LARMIJIRB3VGAFOHANCNFSM5TWVAJFA . You are receiving this because you authored the thread. https://github.com/notifications/beacon/AKQXAPTJ2VOWIHHMSFGV3PLVGAFOHA5CNFSM5TWVAJFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOIHGVJFQ.gif Message ID: @. @.> >

[izzytwosheds]

Having trouble reproducing this locally. Devices I am trying this on do not have an MPEG2 decoder.

[SecureTVBob]

Is there something I can do here with some additional debug outputs added?

[izzytwosheds]

One possible way I can think of is comparing mediaFormats in CodecUtil.configureMediaFormat for .ts and MP4 inputs. That would let us see what exactly is being sent to encoder.