Incorrect parsing of "meta" box for screen recording file with Quicktime

[vjeux]

I saved a screen recording mp4 file with sound with Quicktime (mp4_with_sound.mov.zip). If you open this file in the filereader.html, you can see this in the console:

[BoxParser] 'meta' box writing not yet implemented, keeping unparsed data in memory for later write
[BoxParser] Box of type '' has a size 1751411826 greater than its container size 134
[BoxParser] Parsing of box 'meta' did not read the entire indicated box data size (missing 126 bytes), seeking forward

[selsamman]

I am getting a similar error with any video shot on the Samsung Galaxy S10. In my case as far as I can tell segmenting still works and the moov box is found and seems reasonable. I am able to stream the segments over WebRTC to MSE in a browser. And BTW this is an absolutely awesome package. Using the latest version of mp4box.js

[0:00:00.001] [ISOFile] Processing buffer (fileStart: 0) [0:00:00.002] [BoxParser] Unknown box type: 'SDLN' [0:00:00.003] [BoxParser] 'SDLN' box writing not yet implemented, keeping unparsed data in memory for later write [0:00:00.003] [BoxParser] Unknown box type: 'smrd' [0:00:00.003] [BoxParser] 'smrd' box writing not yet implemented, keeping unparsed data in memory for later write [0:00:00.003] [BoxParser] Unknown box type: '©xyz' [0:00:00.003] [BoxParser] '©xyz' box writing not yet implemented, keeping unparsed data in memory for later write [0:00:00.003] [BoxParser] Unknown box type: 'smta' [0:00:00.003] [BoxParser] 'smta' box writing not yet implemented, keeping unparsed data in memory for later write [0:00:00.003] [BoxParser] 'meta' box writing not yet implemented, keeping unparsed data in memory for later write [0:00:00.003] [BoxParser] Box of type '' has a size 1751411826 greater than its container size 165 [0:00:00.003] [BoxParser] Parsing of box 'meta' did not read the entire indicated box data size (missing 157 bytes), seeking forward [0:00:00.005] [BoxParser] 'colr' box writing not yet implemented, keeping unparsed data in memory for later write [0:00:00.006] [BoxParser] 'esds' box writing not yet implemented, keeping unparsed data in memory for later write [0:00:00.008] [ISOFile] Done processing buffer (fileStart: 0) - next buffer to fetch should have a fileStart position of 5944036 [0:00:00.008] [MultiBufferStream] 1 stored buffer(s) (3436/5944036 bytes), continuous ranges: [0-5944035] [0:00:00.008] [ISOFile] Sample data size in memory: 0 [0:00:00.008] [ISOFile] Flushing remaining samples [0:00:00.008] [MultiBufferStream] 1 stored buffer(s) (3436/5944036 bytes), continuous ranges: [0-5944035]

[davemevans]

The problem seems to be that the ISOBMFF and QTFF specifications define the meta box differently.

The ISOBMFF spec defines meta as FullBox, having flags and version fields after the box type, and mp4box.js parses it as such.

The QTFF spec defines meta as an atom (functionally identical to an ISOBMFF Box), so mp4box.js parses it incorrectly and gets confused about what comes next.

I'm not sure if QTFF is supposed to be supported in mp4box.js, or what should happen when the two differ in definition.

Likely related: https://github.com/gpac/mp4box.js/issues/120

[vjeux]

@davemevans great investigation work! Is there a way to distinguish which version we are using? If not, we could try to parse using one format and if it doesn't fit the number of bytes try parsing using the second format. It's not ideal but would work.

[bradh]

We could just bail after the ftyp box if we find a major brand for quicktime.

[tobiasBora]

Same error here for video taken with a Samsung XCover:

[0:00:11.885] [BoxParser] Box of type '' has a size 1751411826 greater than its container size 165

[jozefchutka]

I have the same error with video taken with Xiaomi phone and mp4box.js 0.52

[0:00:00.009] [BoxParser] Box of type '����' has a size 1751411826 greater than its container size 225

General
Format                                   : MPEG-4
Format profile                           : Base Media / Version 2
Codec ID                                 : mp42 (isom/mp42)
File size                                : 77.4 MiB
Duration                                 : 31 s 667 ms
Overall bit rate                         : 20.5 Mb/s
Frame rate                               : 30.000 FPS
Encoded date                             : 2023-08-30 10:22:20 UTC
Tagged date                              : 2023-08-30 10:22:20 UTC
com.android.version                      : 13
com.android.manufacturer                 : Xiaomi
com.android.model                        : M2101K6G

[RufaelDev]

Yes I observed the same issue as @davemevans 2 years ago filming on my mobile device (huawei p30 lite), many android phones still output quicktime type metabox, but files do not include the brand in the ftyp box (at least mine didnt).

[RufaelDev]

14496-12 defines meta as FullBox('meta', version = 0, 0), so maybe the best way to parse meta to look at the bytes following meta, if it is zero it is an isobmff box, if it is not it is a new box enclosed in the quicktime meta box (non full box) ? does it make sense ? I do not think the brand in ftyp will work as it is usually not set.

[bradh]

14496-12 defines meta as FullBox('meta', version = 0, 0), so maybe the best way to parse meta to look at the byte following meta, if it is zero it is an isobmff box, if it is not it is a new box enclosed in the quicktime meta box (non full box) ? does it make sense ? I do not think the brand in ftyp will work as it is usually not set.

I'm not sure that will work. What is the byte following meta in your quicktime sample? If its another box, it'll start with a four byte length, and the most likely first byte in that length is 0.

[RufaelDev]

@bradh you need to check 4 bytes, it will be zero for fullbox (isobmff) and non zero for quicktime box (hdlr) (bigendian 32 bit interpretation), sorry typed byte instead of bytes ;-)

[bradh]

Possible. Does quicktime use 0x00 0x00 0x00 0x00 in the same way as the ISO BMFF - to mean "to end of file"?

[RufaelDev]

@bradh I don't believe that syntax is explicitly defined, but my main point here is the difference in syntax between the metaBox from QuickTime and the MetaBox from ISOBMFF. So far no version or flags are specified for the ISOBMFF version so the typical syntax would be sizemeta0x00 0x00 0x00 0x00 for the isobmff and size meta [4 bytes > 0] for the QuickTime MetaBox.

[bradh]

https://issuetracker.google.com/issues/232971800 indicates that Google are going to keep doing this. They suggest a slightly different work around to parsing based on checking for hdlr 4cc.