14496-15 deprecated text about aggregators in Section A.2.1

[lkondrad]

In Section A.2.1 of ISO/IEC 14996-15 the following paragraph mentions 'avc1', 'hvc1', and 'hev1'

Aggregators can be used to group base layer or base view NAL units. If these Aggregators are used in an 'avc1', 'hvc1', or 'hev1' track then an aggregator shall not use inclusion but reference of base layer or base view NAL units (the length of the Aggregator includes only its header and the NAL units referenced by the Aggregator are specified by additional_bytes).

while Annex F defines aggregator NAL unity types only for 'avc2', 'avc4','hvc2', and 'hev2'.

Please consider correcting or removing the paragraph from section A.2.1

regards, Lukasz

[jeanlf]

Correct, the sentence should be removed

Also:

in 5.4.2.1.1, talking about avc1 or avc3

The file format specific structures that resemble NAL units (defined in Annex A) may be present but shall not be used to access the AVC base data; that is, the AVC data shall not be contained in Aggregators (though they may may be included within the bytes referenced by the additional_bytes field) nor referenced by Extractors.

This should be removed

in 6.5.3.1.1

Extractors or aggregators may be used for SVC VCL NAL units in 'avc1', 'avc2', 'avc3', 'avc4', 'svc1' or 'svc2' tracks.

this should be

Extractors or aggregators may be used for SVC VCL NAL units in 'avc2', 'avc4', 'svc1' or 'svc2' tracks.

[dwsinger]

we look for a contribution detailing the fixes...

[yekuiwang]

I will handle this.

[yekuiwang]

I just took a look at this issue. I agree that there is a discrepancy between the mentioned normative texts in 5.4.2.1.1, 6.5.3.1.1, and A.2.1 and the informative tables in Annex F. However, as far as I remember, the mentioned normative texts were following the design intent(s) at the time when the respective parts were developed. I was not involved in the development of the informative Annex F and thus don't know whether there was a clear change of the design intent(s) at that time. If there were no such change of the design intent(s), then I would think that that the mistake is in the informative Annex F, and consequently, it is the tables in Annex F that should be fixed.

Any opinion? In particular, Miska what do you think? @mhannuksela

[yekuiwang]

Hmm, after a bit more thinking, I think there is actually no discrepancy between the mentioned normative texts in 5.4.2.1.1, 6.5.3.1.1, and A.2.1 and the informative tables in Annex F. For example, although Aggregators are not defined for 'avc1', it is allowed to be used in an 'avc1' track to refer to data instead of containing data, thus a legacy 'avc1' file parser would ignore the Aggregators, while new 'avc2' or 'avc4' file parsers can utilize the Aggregators.

Therefore, I don't think the normative texts as mentioned should be removed. It might be good, though, to add a NOTE to clarify this.

[mhannuksela]

Hi @yekuiwang,

I agree that the cited normative texts in 5.4.2.1.1, 6.5.3.1.1, and A.2.1 are not conflicting when it comes to Aggregators.

I agree with the design intent as you stated it:

For example, although Aggregators are not defined for 'avc1', it is allowed to be used in an 'avc1' track to refer to data instead of containing data, thus a legacy 'avc1' file parser would ignore the Aggregators, while new 'avc2' or 'avc4' file parsers can utilize the Aggregators.

Once this design intent is agreed/confirmed by everyone (perhaps in the next MPEG meeting), then we'd need to check that there are no conflicting statements in the text. For example, I'd think Table F.1 ('avc1' and 'avc3' nal_unit_type value assignments) would need to include Aggregator (with the additional constraint that it shall not contain data), whereas currently the nal_unit_type value used for Aggregator is reserved in Table F.1.

Furthermore, the text in 6.5.3.1.1 also refers to extractors in 'avc1' and 'avc3':

Extractors or aggregators may be used for SVC VCL NAL units in 'avc1', 'avc2', 'avc3', 'avc4', 'svc1' or 'svc2' tracks.

It seems to me that the design intent is to disallow extractors in 'avc1' and 'avc3'. We perhaps need to split the sentence to two:

Aggregators may be used for SVC VCL NAL units in 'avc1', 'avc2', 'avc3', 'avc4', 'svc1' or 'svc2' tracks. Extractors may be used for SVC VCL NAL units in 'avc2', 'avc4', 'svc1' or 'svc2' tracks.

[yekuiwang]

Cool! Thanks @mhannuksela for taking a deeper look at this issue. Look forward to seeing a contribution at the next meeting suggesting an exact set of text changes for this; I can work with you together for the potential contribution.

[cconcolato]

@mhannuksela Was this addressed in a contribution?

[mhannuksela]

@cconcolato Yes, m63005, but the MPEG-internal GitLab discussion (https://git.mpeg.expert/MPEG/Systems/FileFormat/NALuFF/-/issues/177) did not reach a conclusion how to modify the standard text.

[cconcolato]

Ok, let's keep this GitHub issue open then and try to reach a conclusion

[cconcolato]

Closing this issue and encouraging participants to bring contributions to reengage the conversation.