MIME profile documenting file transformation and extended brands

[jeanlf]

Files may have structural transformations (e.g. compressed files, partial files) signaled through OriginalFileTypeBox. Other files may have further brand descriptions through ExtendedFileTypeBox.

We however miss a way for the profile parameter to document these transformations and further descriptions. We should define something along what has been done for track transformation.

Something like '.' to separate transformation and '()' to identify etyp:

profile="isoC.iso6(fooz+barz, fooX+barX)"
profile="iso6,dash(fooz+barz, fooX+barX)"

[dwsinger]

not sure I understand the syntax. can you work it through?

[jeanlf]

Sure. In case of a compressed file, using ftyp 'isoC', a OriginalFileTypeBox encapsulating a ftyp with single brand 'iso6' and ExtendedFileTypeBox with brands 'fooz'+'barz' and 'fooX'+'barX', you will get:

isoC.iso6(fooz+barz, fooX+barX)

In case of a file with no OriginalFileTypeBox, but a ftyp with brands 'iso6' and 'dash' and an ExtendedFileTypeBox with brands 'fooz'+'barz' and 'fooX'+'barX', you will get:

iso6,dash(fooz+barz, fooX+barX)

If the file has multiple brands 'brd1' and 'brd2' in the ftyp and encapsulates through OriginalFileTypeBox another ftyp with brands 'brd3' and 'brd4':

brd1,brd2.brd3,brd4

If the OriginalFileTypeBox encapsulates another ftyp with brands 'brd5' and 'brd6':

brd1,brd2.brd3,brd4.brd5,brd6

So '.' always indicates a file transformation (i.e. one OriginalFileTypeBox), and '()' always indicates ExtendedFileTypeBox brands. The ExtendedFileTypeBox can happen anywhere in the file brand transformation, but must appear after the ftyp info.

Example with a single transformation, the ExtendedFileTypeBox being added before the OriginalFileTypeBox: iso6,dash(fooz+barz, fooX+barX).brd2

Example with nested transformation, the ExtendedFileTypeBox being added in the forst OriginalFileTypeBox: isoC.dash(fooz+barz, fooX+barX).brd2

[dwsinger]

Hang on.

Today we have

profiles="aaaa,bbbb,cccc+dddd,eeee+ffff+gggg"

where a,b,c are in the FileType box, and c+d,e+f+g are Extended. Right?

so, imagine we have a file with

Ftyp: M,N Extended: O+P,Q+R+S Original Ftyp: a,b,c Original Extended: c+d,e+f+g

what does the profile string look like? I think you are suggesting

profiles="MMMM,NNNN,OOOO+PPPP,QQQQ+RRRR+SSSS,(aaaa,bbbb,cccc+dddd,eeee+ffff+gggg)"

but...where does the dot come in (and what does the fox say ;-) ?)

[jeanlf]

I cannot find any mention in part12 annex K or in RFC6381 of '+' for profiles, did I miss something?

[dwsinger]

you are right. we talked about it but never wrote it down

[jeanlf]

OK, so let's write it down ;)

'+' in your example is '()' in mine, indicating extendedFileType brands as a list.

So for your example:

Ftyp: M,N
Extended: O+P,Q+R+S
Original Ftyp: a,b,c
Original Extended: c+d,e+f+g

We would have with the proposed syntax:

profiles=M,N(O+P,Q+R+S).a,b,c(c+d,e+f+g)

• '.' marks brand transformation (OriginalFileTypeBox)
• '()' marks extendedFileTypeBox info
• '+' is used for combination of brands in extendedFileTypeBox

Is this more clear ?

[dwsinger]

ah, so in your syntax, "." separates M,N(O+P,Q+R+S) and a,b,c(c+d,e+f+g), whereas I thought it was somehow attaching a to (O+P,Q+R+S)

and I am not sure why I need both the () and the +, can you elaborate? It looks as if (O+P,Q+R+S) is somehow associated with N

[jeanlf]

the + is to list the brands in one TypeCombinationBox, the '(,)' is to separate the different TypeCombinationBoxes.

If we don't use '(' we don't have a clear separation of ftyp vs etyp but that's not a big issue I agree. For your example this would give:

profiles=M,N,O+P,Q+R+S.a,b,c,c+d,e+f+g

The '.' is a marker for OriginalFileTypeBox, it does not "attach" it to the preceeding brand. I also thought about using '()' to encapsulate OriginalFileTypeBox, but we might end up with nested parenthesis, not really nice.

[cconcolato]

Some thoughts: profiles=M,N(O+P,Q+R+S).a,b,c(c+d,e+f+g) is confusing:

• having c in an Ftyp and c+d in the extended box does not make sense
• the parentheses give the impression that O+P,Q+R+S is applied to N.
• if you split the string on commas, you end up with M then N(O+P then Q+R+S).a then b then c(c+d then e+f+g) which is not what is intended
• Implementations might complain if the length of the tokens between commas is greater than 4 chars (also applicable to the + sign).
• There is a big difference between 'Extended' and 'Original'. For Extended, it is just the signaling that changes. If you ignore it, you can still process the file if you understand the other values. For Original, the file cannot be used unless you understand the brand.
  • It's unclear to me what the Original is associated with, the set M,N,O+P,Q+R+S or just Q+R+S (doesn't make sense)

I see several options:

• a new MIME type, e.g. when transformation is compression you could have video/mp4+gz and just list the untransformed profiles. (does not work for cascaded transforms)
• 2 parameters: profiles=M,N,O+P,Q+R+S untransformedProfiles=a,b,c,c+d,e+f+g (does not work for cascaded transforms)
• N parameters: profiles=M,N,O+P,Q+R+S untransformedProfiles1=a,b,c,c+d,e+f+g (repeat with untransformedProfiles

[jeanlf]

The real question here is whether we need backward compatibility or not for profiles.

For OriginalFileTypeBox, if one fails to parse the profilesparameter that means the OriginalFileTypeBox is not supported, so no harm done.

For ExtendedFileType it is trickier indeed, because an implementation could support one of the listed brand from ftyp but not the full profiles (whatever the syntax is), and fail to load a file it would have correctly loaded without the extended brands...

Also I think we need to support cascaded transformations, but I don't really like having one new param per transformation ...

[dwsinger]

my suspicion is that the a+b syntax for extended types is the safest we'll get. the most likely client behavior is to separate on "," and see a+b as an unrecognized type.

somehow, we need to have them avoid seeing the brands in the originalfiletype as recognized, though

I was going to suggest parentheses, which can be nested, for originalfiletype, but if today's client splits on commas, it'll see

profiles=M,N,O+P,Q+R+S,(a,b,c+d,e+f+g)

and I fear it'll split on commas and see b as a brand. (it'll see (aaaa as an invalid brand, as well)

[mhannuksela]

Since this change may affect the legacy parsers, a safe option is not to overload the profiles parameter with any values contained in the ExtendedTypeBox or OriginalFileTypeBox. Instead, I’d propose defining two new optional MIME parameters, e.g. combination-profiles and original-profiles, which have one-to-one mapping of file-level ExtendedTypeBox and OriginalFileTypeBox, respectively.

combination-profiles is a comma-separated list of entries in the file-level ExtendedTypeBox, each entry being a plus-separated list of the values contained in the respective TypeCombinationBox.

Example, where the file is compatible with a combination of O and P brands and a combination of O and R brands: combination-profiles=O+P,O+R

original-profiles is a comma-separated list of the entries of the file-level OriginalFileTypeBox, where the entries are derived from the OriginalFileTypeBox with the following rules:

• If the OriginalFileTypeBox contains a FileTypeBox, then the comma-separate entries are formatted like in the profiles parameter;
• If the OriginalFileTypeBox contains an ExtendedTypeBox, then the comma-separated entries are formatted like in the combination-profiles parameter;
• If the OriginalFileTypeBox contains a second OriginalFileTypeBox, the comma-separated entry starts with ”(” and ends with ”)” and these rules for comma-separated entries apply recursively for the boxes contained in the second OriginalFileTypeBox.

Example, where a file compatible with a combination of iso8 and relo brands has been further transformed with box compression i.e. complies with comp brand:

profiles=comp
original-profiles=iso8+relo

If the above file is further transformed with an imaginary trns brand, it would contain the following:

profiles=trns
original-profiles=comp(iso8+relo)

[dwsinger]

I suspect that allowing combination profiles with + is mostly backwards compatible, as at worst a parser will reject a+b+foo as an unrecognized brand that's more than four characters.

But I agree that trying to pack original brands in there seems a stretch

[dwsinger]

we need to consider what happens when the top-level has multiple brands or even combination brands…

[dwsinger]

add the typecombination + syntax to the WD and keep thinking about the transformation case...

[mhannuksela]

The typecombination + syntax as discussed above has been included in the ISOBMFF 8th Edition, subclause K.4.

[cconcolato]

Closing this issue as the text is integrated in the 8th edition.