obu_size field usage

[bilboed]

Section 3.1 has something confusing in it :

• An AV1 video stream shall have the low overhead byte stream format as defined in [AV1].

And then

• An OBU may contain the obu_size field. For applications that need easy conversion to MP4, using the obu_size field is recommended.

Except that the AV1 specification state:

This specification defines a low-overhead bitstream format as a sequence of the OBU syntactical elements defined in this section. When using this format, obu_has_size_field must be equal to 1.

Therefore section 3.1 goes against the AV1 specification, obu_size field must be present and is not optional.

[bilboed]

By extension, if the low-overhead bitstream format mandates the size to be present, it provides an easy way to scan through the various OBU. Therefore there is no need for the custom "start-code based" format, which was added in #5 without any rationale or context

[cconcolato]

Thanks for filing this issue. You're raising an interesting problem which goes beyond this MPEG-2 TS binding spec. For example, the AV1-ISOBMFF spec says (https://aomediacodec.github.io/av1-isobmff/#sampleformat):

each OBU SHALL follow the open_bitstream_unit Low Overhead Bitstream Format syntax as specified in [AV1]. Each OBU SHALL have the obu_has_size_field set to 1 except for the last OBU in the sample, for which obu_has_size_field MAY be set to 0, in which case it is assumed to fill the remainder of the sample,

So one can consider that what's stored in an MP4 file is not necessarily a "valid" low-overhead bitstream format (if the last obu_has_size_field is set to 0), but in that case, it's the job of the MP4 demuxer to update the header if needed. For example, in a particular demuxer/decoder integration, having the total sample size could be sufficient. That said, I think the AV1-ISOBMFF spec could be clarified.

I think we can approach the carriage in MPEG-TS in a similar way, but we definitely need to make the spec clearer. If you care about compliance to the low-overhead bitstream format of what comes out of the demuxer, then the demuxer should ensure that the size fields are there.

Regarding the redundancy between the size field and the start codes, this was understood by the group when the start code approach was discussed. The resulting text (which should be improved) is a compromise, and hence is not perfect.

Regarding the need for start codes, it was discussed at length internally in the working group (and probably should have been better captured for the general public). The initial issue was that the absence of start codes required complex processing in hardware and secure pipelines to determine the boundaries of frames and OBUs in general. Several approaches have been envisaged and discussed with several software and hardware implementors and the use of start codes was again the final compromise.

[bilboed]

Thanks for your feedback !

Yes, I had noticed the issue in av1-isobmff also. It's unfortunate, and should indeed be clarified.

I completely agree some "adaptation" is required between codec bitstream and container format integration, and is perfectly understandable, but it should try to minimize the modifications of the bitstream to a minimum. My concern isn't just about having the bitstream ingested by (or received from) a MP4 container, but by any other kind of "processing" that complies with the official AV1 specification. Some kind of "common" bitstream exists, and is one of the two in the AV1 specification.

Could the findings of that working group you mention (which one is it ?) be made available so that there isn't duplication of work in this effort ?

[cconcolato]

The AOM group in charge of producing the AV1-ISOBMFF or the AV1-MPEG-2TS spec is the "Storage and Transport Format" Working Group.

[cconcolato]

Can we close this issue (and discuss the remaining aspects in your other issue)?

[bilboed]

sure