Packaging HEVC/AAC TS live stream supported?

[hariszukanovic]

System info

Operating System: CentOS Linux release 7.5.1804 (Core) Shaka Packager Version: 2.2.1

Issue and steps to reproduce the problem

Packager Command: /root/shaka-packager "input=udp://232.100.100.34:1234?reuse=1&interface=0.0.0.0&timeout=5000000,stream=video,stream_selector=0,bandwidth=15000000,init_segment=dash/cenc/init_v.m4s,segment_template=dash/cenc/chunkv\$Number\$.m4s,output_format=mp4" "input=udp://232.100.100.34:1234?reuse=1&interface=0.0.0.0&timeout=5000000,stream=audio,language=eng,stream_selector=2,bandwidth=99000,init_segment=dash/cenc/init_a_eng.m4s,segment_template=dash/cenc/chunk_aeng\$Number\$.m4s,output_format=mp4" --segment_duration 10 --generate_dash_if_iop_compliant_mpd=1 --minimum_update_period 10 --time_shift_buffer_depth 60 --mpd_output manifest.mpd

Extra steps to reproduce the problem? No

What is the expected result? To start packaging...

What happens instead? [1018/154409:INFO:demuxer.cc(88)] Demuxer::Run() on file 'udp://232.100.100.34:1234?reuse=1&interface=0.0.0.0&timeout=5000000'. [1018/154409:INFO:demuxer.cc(160)] Initialize Demuxer for file 'udp://232.100.100.34:1234?reuse=1&interface=0.0.0.0&timeout=5000000'. [1018/154410:ERROR:demuxer.cc(204)] Not implemented reached in shaka::Status shaka::media::Demuxer::InitializeParser() [1018/154410:ERROR:packager_main.cc(517)] Packaging Error: 4 (UNIMPLEMENTED): Container not supported.

<Please attach the input files or email to shaka-packager-issues@google.com.> Short capture of the input in.zip

Additional Info: This is the free NASA TV channel taken down from sattelite (no transcoding)

ffprobe udp://232.100.100.34:1234 Input #0, mpegts, from 'udp://232.100.100.34:1234': Duration: N/A, start: 19210.293489, bitrate: N/A Program 505 Stream #0:0[0x13bb]: Video: hevc (Main) ([36][0][0][0] / 0x0024), yuv420p(tv, bt709), 3840x2160 [SAR 1:1 DAR 16:9], Closed Captions, 59.94 fps, 59.94 tbr, 90k tbn, 59.94 tbc Stream #0:1[0x13bc]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 64 kb/s Stream #0:2[0x13bd]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 90 kb/s

[hariszukanovic]

From the official table of supported codecs and containers, my understanding is that H265 (HEVC) inside MPEG2-TS is actually supported and I think the live stream in question contains exactly this combination.
Would that indicate a bug? It is strange that the error shown is a clear "Container not supported", so perhaps I am failing to identify the container of this stream correctly?

[hariszukanovic]

Additionally, if I pass the stream through ffmpeg... just copying and outputing in MPEGTS, ./ffmpeg -i udp://232.100.100.34:1234 -map 0:0 -c copy -f mpegts "udp://239.0.0.1:1234?reuse=1&timeout=10000000&overrun_nonfatal=0&buffer_size=16777216&fifo_size=286720"

..shaka-packager understands the stream and begins packaging. I suppose the container as received from the satellite in fact is not MPEG2-TS for some reason...

[hariszukanovic]

On the other hand, now while packaging I receive quite frequently other errors. Sometimes at the startup, sometimes in the middle of packaging after a varying amount of time passed with no apparent regularity... Output dash seems to work until an error occurs.

This one, mostly only at startup [1019/162102:INFO:demuxer.cc(88)] Demuxer::Run() on file 'udp://239.0.0.1:1234?reuse=1&interface=0.0.0.0&timeout=5000000'. [1019/162102:INFO:demuxer.cc(160)] Initialize Demuxer for file 'udp://239.0.0.1:1234?reuse=1&interface=0.0.0.0&timeout=5000000'. [1019/162102:ERROR:packager_main.cc(517)] Packaging Error: 8 (PARSER_FAILURE): Cannot parse media file udp://239.0.0.1:1234?reuse=1&interface=0.0.0.0&timeout=5000000

This one, after a while... while packaging [1019/162112:INFO:demuxer.cc(88)] Demuxer::Run() on file 'udp://239.0.0.1:1234?reuse=1&interface=0.0.0.0&timeout=5000000'. [1019/162112:INFO:demuxer.cc(160)] Initialize Demuxer for file 'udp://239.0.0.1:1234?reuse=1&interface=0.0.0.0&timeout=5000000'. [1019/162115:ERROR:h26x_byte_to_unit_stream_converter.cc(62)] H.26x byte stream frame did not begin with start code. [1019/162115:ERROR:packager_main.cc(517)] Packaging Error: 8 (PARSER_FAILURE): Cannot parse media file udp://239.0.0.1:1234?reuse=1&interface=0.0.0.0&timeout=5000000

[rkuroiwa]

For line

[1018/154410:ERROR:demuxer.cc(204)] Not implemented reached in shaka::Status shaka::media::Demuxer::InitializeParser()

This means that the packager could not determine the container format of the stream. As you mentioned it might not be in TS if running it thru ffmpeg resolves this problem.

For

[1019/162102:ERROR:packager_main.cc(517)] Packaging Error: 8 (PARSER_FAILURE): Cannot parse media file udp://239.0.0.1:1234?reuse=1&interface=0.0.0.0&timeout=5000000

and

[1019/162115:ERROR:h26x_byte_to_unit_stream_converter.cc(62)] H.26x byte stream frame did not begin with start code.

I think we need to add better logging. e.g. dump the first few bytes that the packager saw, instead of just logging that it wasn't a start code. That said, is it possible that there is a packet loss?

I will try to take a look, with the zip file that you've provided :)

[hariszukanovic]

The zip file is just a random capture of the input. I have not actually tried to reproduce the problem with it as input.

There should not be a package loss, but of course it is always possible. It is a stream at ~15Mbit. It might also be a good idea to clearly log the loss as detected using the CC in MPEGTS.

Additionaly, not to tease... but, using the same input I output the dash directly from ffmpeg and it works very well.

Is it possible that closed captions in video stream create the problem of "container not recognised"?

On Fri, 19 Oct 2018, 22:16 rkuroiwa, notifications@github.com wrote:

For line

[1018/154410:ERROR:demuxer.cc(204)] Not implemented reached in shaka::Status shaka::media::Demuxer::InitializeParser()

This means that the packager could not determine the container format of the stream. As you mentioned it might not be in TS if running it thru ffmpeg resolves this problem.

For

[1019/162102:ERROR:packager_main.cc(517)] Packaging Error: 8 (PARSER_FAILURE): Cannot parse media file udp:// 239.0.0.1:1234?reuse=1&interface=0.0.0.0&timeout=5000000

and

[1019/162115:ERROR:h26x_byte_to_unit_stream_converter.cc(62)] H.26x byte stream frame did not begin with start code.

I think we need to add better logging. e.g. dump the first few bytes that the packager saw, instead of just logging that it wasn't a start code. That said, is it possible that there is a packet loss?

I will try to take a look, with the zip file that you've provided :)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/google/shaka-packager/issues/492#issuecomment-431486324, or mute the thread https://github.com/notifications/unsubscribe-auth/ABLXKHEMfj-FhtnimYzUbZqzwiFhBnEnks5umjMVgaJpZM4XtfO8 .

[rkuroiwa]

Found out several things.

1. The TS file in the zip file doesn't cause the "container not recognized" error, so I can't figure out what the cause is.
2. There seems to be an off by 1 error. I've hacked things locally so that the packager exits successfully, but I'm not sure if this is a bug in the packager or in the stream. So I have to read the spec and code more carefully. This is likely causing the "Packaging Error: 8" problem since I see it in the logs.

Note: If I made this line

auto pos = es_queue_->tail();
if (pos != 0) {
  pos--;
}
std::pair<int64_t, TimingDesc>(pos, timing_desc));

The packager exits fine.

[hariszukanovic]

The capture command output the MPEGTS rewriting it. I am not sure if there is a way to capture without remuxing with ffmpeg

FFmpeg detectes this input MC stream as MPEGTS. I wonder how it would work at all in ffmpeg (as it in fact demuxes it as MPEGTS) if it actually weren't MPEGTS.

Can you work with a wireshark or similar dump of incoming UDP stream to do your tests?

[rkuroiwa]

I think I've done it once with a static content with clvc? But I can't remember the commands. I have to look it up again. It may be because the packager just looks at the first few bytes when it starts. So if the start doesn't happen to be a TS packet, then it gives up. FFmpeg might be doing a little more sophisticated guessing.

About off-by-1 error: This may have to do with how the packager finds NALU start code and determine the "position" of the NALU.

I think the problem is with this logic

I'm going to reference the H.265 spec but there is similar guideline in H.264 as well.

• The 0 byte at such position could be the trailing_zero_8bits (in B.2.1) of the previous NALU.
• B.3 recommends to "discard" start codes when in a "decoding process". Altho the packager doesn't decode, it should probably not use the start code's position as the NALU's start position.

So the actual start code may be 0x000001 (3 bytes) but the packager thinks that it's 4 bytes (0x00000001) which is causing the off by 1 error.

[kqyang]

@hariszukanovic I have updated the code to dump the buffer on 'Container not supported' error. See #505 for details. It should help us isolate where the problem is for that error. Would you mind to run the latest packager on your input and send the buffer to us if you see the error again?

[rkuroiwa]

With further investigation, we found out that the stream may be non-spec compliant.

What the spec allows

The TS spec (form 2015) mentions this for AVC, and AFICT from searching the internet, there is a similar clause for HEVC in the latest revision of the spec.

For AVC video streams conforming to one or more profiles defined in Annex A of Rec. ITU-T H.264 | ISO/IEC 14496-10 video, if a PTS is present in the PES packet header, it shall refer to the first AVC access unit that commences in this PES packet. An AVC access unit commences in a PES packet if the first byte of the AVC access unit is present in the PES packet.

This allows starting an AU in the middle of a PES packet. (IIUC) This basically allows "packing" as many byte as possible to TS packets and not "waste" any bandwidth by padding the TS packet at the end of a PES packet.

Algorithm to process the TS stream

Due to the clause in the spec, the packager keeps around the data from the previous PES packet; by concatenating the data with the current PES packet, it determines an AU.

Also note that until it sees the next AUD NALU (or some other strong signal that it is a new AU) the packager cannot determine that an AU has ended. This is just the nature of NALU streams.

Input Stream

The in.ts file's video PES packets' pay load always start with an AUD NALU, i.e. it starts with (hex) 00 00 01 46. Note that the start code is 3 bytes (0x000001). The rest of the stream has 4 byte start code (0x00000001). Also, every video PES packet end with a 0x00.

Problem

As mentioned above, every PES end with a 0x00 and every PES start with 0x000001. This forms a 4 byte start code. Which means the AU starts from the previous PES.

Since the spec says that the PTS only applies to the first AU in the PES, there is no PTS for the second AU, and AU without a PTS is invalid.

OTOH, an AVC/HEVC Annex B stream is allowed to have as many 0s at the end of a NALU. So the 0x00 at the end of PES packets may be such 0s. But it's hard to believe so because all the other NALU start codes in the stream use the 4 byte version (0x00000001).

Possible solution

(Assuming that the intent of the stream is to have 1 AU in a PES)

Note that the packager can process the first PES because 3 byte start code is a valid start code. But due to the algorithm, it keeps around the data from the previous PES to process the current (second and on) PES.

1. Find the AUD NALU in the current PES and assign the PTS.
2. Find all the other whole NALUs in the current PES. Mark all the data that has been processed.
3. Process the data between the AUD NALU (from 1) and the data that has not been processed from the previous PES.
4. Emit the AU that started in the previous PES as a frame.

[kqyang]

@rkuroiwa Thanks for the in-depth analysis of the problem! I agree that it is likely a bug in their content generator to end the PES with "00" and start the PES with "00 00 01" (and they use 4 byte start code everywhere else). I don't think the result content is non-spec compliant though, since the spec allows appending 00s to the end of Annex.B byte streams.

.. xx xx 00 | 00 00 01 xx xx xx .. xx xx 00 | 00 00 01 xx xx xx ..

A spec compliant approach to the above content is to keep the ending 00s in the previous access unit, i.e. don't include the 00s in the previous PES to the current access unit, like what FFmpeg is doing. This requires the parser to be aware of the PES boundary.

That said, the ending 00s are not useful and will be discarded anyway after converting to RBSP (the actual ending 00 in the RBSP is appended with 03 when start code emulation prevention is applied). So it is fine to either keep or discard the ending 00s in the access unit.

Find the AUD NALU in the current PES and assign the PTS.

PTS is associated with the access unit.

So do you want to (1) Keep the current NALU searching logic and consider the AUD NALU with start code in previous PES packet the starting of the current access unit, or (2) Revert the |start_code_offset| backoff if it is an AUD (or any other NALU that can start an access unit) if it crosses PES boundary?