Machine-readable representation of boxes

[stevn]

I'm currently working on a small MP4 parser and analyzer. It currently reads a JSON representation of the structure of known box types, then parses the MP4 file and dumps the information.

Now I was wondering if there is any machine-readable representation of "all" boxes or at least the most important ones. Registering all of them would be a lot of work and kind of impossible for me.

[baumanj]

Per ISOBMFF (ISO/IEC 14496-12:2020) § 4.2.1 Object syntax conventions:

The definitions of objects are given in the syntax description language (SDL) defined in ISO/IEC 14496-1.

Here's an example from HEIF (ISO/IEC 23008-12:2017) (which is freely available) § 6.5.10.2:

aligned(8) class ImageRotation
extends ItemProperty('irot') {
       unsigned int (6) reserved = 0;
       unsigned int (2) angle;
}

In general, the box definitions are spread across many different specs (though mp4ra.org is very helpful for finding which one). I'm not aware of a resource that has them collected into one place.

[podborski]

Check out https://github.com/mp4ra/mp4ra.github.io/tree/dev/CSV

[stevn]

Thanks for the links and the SDL info! I had a look at some SDL specs and really liked how precise they are. It's exactly what I would need, but centrally for all boxes. So it would be great to have a repository of these SDL definitions. It looks like that hasn't been done yet, so what I'm searching for might not exist yet...

[dwsinger]

We've been looking at using Kaitai but none of the input syntaxes it supports seem human-readable enough to embed in specs. I fully support moving to something that is both verifiable (i.e. we can run a syntax checker) and could build a parser/verifier (as Kaitai does), as that would at least automate first-level (structure) verification, though of course there are many other mandates that have to be respected (matching counts, for example).

No-one has yet found the time to bring the Flavor SDL parser up to date, nor integrate it with Kaitai (or write something new); or propose another way ahead. But we are very interested in doing better.

I apologize for the lax nature of the SDL; we have lapsed into the habit of copying other declarations and editing, without verification, which is a way of introducing genetic mutations that persist.

[vectronic]

There is also the MPEG standard 23001-5 Bitstream Syntax Description Language which uses XML. I am not advocating using this, but would be interested to hear people's opinion on it versus SDL. The argument for usage of BSDL vs SDL in documenting bitstreams (as far as I have read) is simply:

"it is stable and defined by an international standard"

which I gather refers to the fact that in contrast, SDL exists somewhat organically across a few standards (14496-1, 13818-1 and 11172-1) and an ageing implementation (Flavor).

My personal aim (dream) would be to document SDL as a formal grammar and then extend it with some useful features such as type annotations for documentation (a-la JavaDoc), some form of 'include' and possibly versioning etc. This would then make it suitable for use in developing a repository of bitstream formats (not just ISOBMFF but also other formats like MXF) together with generated tables, diagrams, standards references etc.

Beyond the readability constraints of Kaitai input definitions, I note it doesn't support serialisation which would be one of my use cases. Therefore a front-end to go from SDL to Kaitai doesn't fit for me.

Flavor's support of XML has no interest for me and my use case is to generate Rust code, so given my proposed SDL extensions I am reticent to start with Flavor.

Hence I keep ending up with the thought of a new SDL based parser and serialiser/deserialiser generator (written in Rust) and targeting a Rust runtime (but built to be extensible). However this is no small undertaking and therefore I am letting this thought process brew for a while.... Would be happy to hear any opinions...

[cconcolato]

Thank you @vectronic for your participation. Yes, we should maybe look at BSDL, if only to understand if it can work. Can you clarify what you mean by:

Kaitai [...] doesn't support serialisation

[vectronic]

"Kaitai [...] doesn't support serialisation" => http://doc.kaitai.io/faq.html#writing

[edrthomas]

For information, MPEG is working on creating a standalone specification for SDL. This will allow us to easily extend the language via amendments. So far, SDL was part of the very large 14496-1 and thus virtually impossible to maintain and extend.

I also hope we can develop a SDL parser along with this new spec. This would primarily be intended for MPEG experts to verify spec text when writing SDL syntax but hopefully this parser could also be useful for a broader audience.

On the long term, automatic generation of a binary writer and parser is also a goal I share for SDL-based spec, but the path towards this is still to be defined, i.e. Kaitai (if serialisation eventually comes), using another parser, writing a new parser, etc...

[vectronic]

@edrthomas Excuse the direct message via a GitHub issue, not sure how else to get in touch...

I am member of the UK's MPEG ISO national body. I have seen the WD of 14496-34 Syntactic Description Language and wanted to get involved. Is this being done under WG-3 Systems? Are there planned meetings for this at the MPEG 140 meeting in Mainz?

[podborski]

@vectronic Yes the work is done in WG3 and I guess some input could be expected in Mainz. The WD is also publicly available here.

[cconcolato]

The group has the goal to produce description of the box (using SDL) that will be machine processable but there is no clear timeline yet. We should also consider providing the syntax as an electronic attachment (much like other specs like DASH provide the XSD).