search

MPEGGroup / MIAF

Official MPEG repository to discuss issues on MIAF (ISO/IEC 23000-22)
2 stars 0 forks source link

Consider relaxing rules on mandatory presence of `mif1` brand #5

Closed cconcolato closed 3 years ago

cconcolato commented 3 years ago

The current version of MIAF Amd 2 says:

The FileTypeBox shall contain, in the compatible_brands list, the 'mif1' brand (specified in ISO/IEC 23008-12).

Having it mandatory when derived specifications (AVIF) already include conformance to MIAF and already define a brand to signal that seems unnecessarily redundant. The only benefit seems to be that codec-agnostic HEIF/MIAF processors can know about the MIAF-compliance of the file. Is it a strong argument to making it a "shall"? Should it be a "should"?

baumanj commented 3 years ago

Making this a "should" would be helpful since consumers of AVIF files which lack the mif1 brand but are otherwise valid are forced to choose between rejecting usable files and correctly implementing the spec. Likewise producers are forced to choose between increasing file size in a way that doesn't benefit them and correctly implementing the spec.

joedrago commented 3 years ago

I'd certainly prefer switching it away from "shall" in this case. We're in a weird spot with interop between Chrome (which uses libavif) and Firefox, where Firefox's (newer) implementation is demanding the presence of this brand due to this clause in the standard, but libavif doesn't.

Any file created with libavif always emits the mif1 and avif brands (and occasionally avis), but the reader doesn't enforce the presence of mif1, and instead just looks for avif or avis. It seems that there are alternative implementations of AVIF packaging (in the same spirit as tools like pngcrush) that try to strip as many unnecessary pieces of data as possible, and I believe they're omitting mif1 because the avif ftyp signaling happens to be sufficient for libavif/Chrome right now.

To solve the interop issues between Firefox and Chrome, we have two paths:

I'd prefer the second choice, as I feel like requiring those additional 4 bytes in ftyp despite already signaling that the file is a legal avif or avis just adds extra bytes to the payload for no significant gain, and introducing this error path to Chrome is going to cause a lot of pain. I would only want to take this approach though if we knew that an addendum to MIAF was pending which laxed this restriction.

cconcolato commented 3 years ago

Note that HEIF also mandates mif1

10.2.1 'mif1' structural brand 10.2.1.1 Requirements on files Files shall contain the brand 'mif1' in the compatible brands array of the FileTypeBox.'

dwsinger commented 3 years ago

readers are always free to give a try to decoding any file, of course. the hope of the brands is to make diagnosis of likelihood of success faster and easier

there are a lot of potential and even specified (e.g. MPEG-21) uses of the file-level meta-box that aren't HEIF. I would hope a brand from the HEIF spec. would be helpful?

cconcolato commented 3 years ago

@dwsinger one brand should be sufficient. If the semantics of AVIF mean that avif includes mif1, why repeat mif1?

joedrago commented 3 years ago

I recognize the rationale for having multiple signals in ftyp that say "if you only know how to decode xxxx, I should work". In this case, I imagine mif1 being present specifically means if you wrote a parser that could only handle mif1 in specific (and no additional features), having the brand here is nice.

That said, I think omitting a brand should simply lock you out from that particular breed of parsers, and nothing more. By omitting mif1 from an AVIF, you lose the ability for that file to be parsed by something that can only parse mif1, but if you're only targeting avif/avis parsers, you knowingly make that tradeoff for a smaller footprint, right? Demanding that a second brand is present for the file to be a valid MIAF-derived format seems excessive.

dwsinger commented 3 years ago

@joedrago mif1 is a HEIF-level brand, the most basic brand that defines structural compatibility.

@cconcolato the value is, as you say, indicating to structural tools (e.g. that a tool that removed un-referenced bytes from mdat/idat, that re-orders the media data so that the primary item's data is early, thumbnail media data is in ladder order) that they are OK.

I'd be fine with 'should' having said that.

wantehchang commented 3 years ago

I support Jon Bauman and Joe Drago's proposal of making this a "should".

cconcolato commented 3 years ago

@dwsinger I also think HEIF should be updated (replacing 'shall' with 'should'). That seems in line with how ISOBMFF brands are done (the base isom is not mandatory) and in line with how mif2 was defined in HEIF AMD2:

Files containing the brand 'mif2' in the compatible brands array of the FileTypeBox shall conform to the constraints defined in this subclause. The brand 'mif2' requires support for all file features of the 'mif1' brand.

Clearly, it is meant to be possible to only signal mif2 if you want, and omit mif1.

joedrago commented 3 years ago

readers are always free to give a try to decoding any file, of course. the hope of the brands is to make diagnosis of likelihood of success faster and easier

This comment got me thinking about this in the context of the Robustness Principle, and how different avenues/usages of AVIFs should behave.

I think if we're considering libavif to be the "official" library for writing AVIFs, then of course it should always emit files that match the spec exactly. In this specific context (the presence of mif1), I believe it does. If we're talking about validators like ComplianceWarden, of course that should also be quite precise/pedantic.

But what about readers / decoders? I feel like there is some wiggle room here; perhaps some sort of strict mode, which shouldn't be on for a viewer (like a web browser). How much burden do we place on a browser to be the arbiter of standards compliance? I recognize the potential slippery slope here, but if a reader can/should always just take their best-effort shot at a file, perhaps this whole conversation is moot? Should we instead focus efforts on some sort of strict/pedantic/validator mode for libavif and mp4parse, and simultaneously not enable them for their associated browsers?

leo-barnes commented 3 years ago

We are not opposed to changing the requirement on mif1 from a "shall" to a "should" in principle, but there are multiple HEIF implementations already present that may not accept files like these. Apples HEIF parser will for instance not accept files like these.

Can the wording be changed so that it remains a "shall" for HEIF and MIAF, but is a "should" for other extensions like AVIF?

dwsinger commented 3 years ago

@leo-barnes I think AVIF builds on HEIF and it's not common to see derived specs relaxing mandates of the specs they build on

cconcolato commented 3 years ago

Maybe we could:

leo-barnes commented 3 years ago

@dwsinger

@leo-barnes I think AVIF builds on HEIF and it's not common to see derived specs relaxing mandates of the specs they build on

Right, I suspected that might be the case. Would it be possible to re-word the HEIF spec to say that it's a "shall" unless some other brand explicitly implies that it's present? We could then change AVIF to explicitly state that the presence of avif implies the presence of mif1. You would still need to explicitly have msf1 there for avif sequences unless there's a specific AVIF brand for that which can be changed the same way to imply msf1

If that's not possible, then I think @cconcolato's proposal might be good. We should probably make it a shall for the original codecs (HEVC, AVC, JPEG) since there could be parsers out there that rely on them having mif1 in the brands.

baumanj commented 3 years ago

As far as I can tell the situation with mif1 being required rather than suggested is unusual. For brands that imply ancestor brands (e.g. mif1 -> miaf, miaf -> iso8, etc.) the inclusion of the more general type seems to be a "should", not "shall", so getting mif1 in line with that practice seems preferable from a consistency standpoint.

The only downside to omitting mif1 in an AVIF I can see is that tools which grok mif1 but are unaware of avif won't necessarily be able to process these files. Considering such tools would have limited utility relative to anything that can actually display or write AVIF, I'd say that's a reasonable trade-off for the authors of the individual files to make: greater compatibility vs saving a few bytes.

dwsinger commented 3 years ago

FYI, we are going to fix the HEIF spec. to reverse the sentence; not that the mif1 brand is required, but if it's present, it indicates that the file conforms to a set of requirements, and that readers claiming to implement the brand have a certain minimal set of capabilities.

If we make that change, the 'shall' will disappear from the HEIF spec. It's then a question of whether AVIF, MIAF etc. mandate it or not. Does that help?

wantehchang commented 3 years ago

In addition to the mif1 brand, ISO/IEC 23000-22:2019(E) says, in Section 10.1 General:

Files conforming to the general restrictions in clause 7 shall include the brand 'miaf' in the compatible_brands in the FileTypeBox.

ISO/IEC 23008-12:2017(E) says in Section 10.3.1.1 Requirements on files:

Files shall contain the brand 'msf1' in the compatible brands array of the FileTypeBox.

and

It is required that 'iso8' is present among the compatible brands array.

cconcolato commented 3 years ago

In addition to the mif1 brand, ISO/IEC 23000-22:2019(E) says, in Section 10.1 General:

Files conforming to the general restrictions in clause 7 shall include the brand 'miaf' in the compatible_brands in the FileTypeBox.

This is fixed in MIAF Amd2.

leo-barnes commented 3 years ago

@dwsinger

FYI, we are going to fix the HEIF spec. to reverse the sentence; not that the mif1 brand is required, but if it's present, it indicates that the file conforms to a set of requirements, and that readers claiming to implement the brand have a certain minimal set of capabilities.

If we make that change, the 'shall' will disappear from the HEIF spec. It's then a question of whether AVIF, MIAF etc. mandate it or not. Does that help?

Discussed this some more internally. This still feels like a pretty risky change given there will be devices out there that currently support HEIF that most likely won't receive SW updates.

Should we then at least mandate that mif1 "shall" be present when using the MIAF HEVC/AVC/JPEG profiles?

cconcolato commented 3 years ago

This has been fixed in FDAM2, by saying:

The FileTypeBox should contain, in the compatible_brands list, the 'mif1' brand (specified in ISO/IEC 23008-12). Some MIAF profiles mandate the presence of this brand.

and then for HEVC/AVC/JPEG profiles, the following was added:

As permitted in 7.2.1.2, this profile requires the brand 'mif1' to be present in the FileTypeBox.