fpv-wtf / wtfos-configurator

Configurator for wtfos, with built in margerine
GNU Affero General Public License v3.0
42 stars 16 forks source link

MP4 Parser/Writer Improvements #278

Open Knifa opened 1 year ago

Knifa commented 1 year ago

Issue to track general improvements needed for the MP4 parser/writer used for the OSD overlay tool.

1. Break MP4 parser/writer out into it's own library.

I wrote this because there weren't any good ones out there for this purpose (video processing pipelines). The streaming write aspect is particularly useful since the other alternatives (mp4box.js, e.g.) buffer the entire thing into memory at once so you hit memory limits fast.

The rest of these issues would then fall into it's own repository in that but until then...

2. AvcC Writing / Stream Reader

The AvcC struct is given to us by the encoder on the first frame, and then we read the struct out directly and turn it into a Proper Object (tm) before writing it back later. It would be nice to have a generalised stream reader for this, like we have for the file reader.

e.g., instead of:

configurationVersion: view.getUint8(0),
      profileIndication: view.getUint8(1),
      profileCompatibility: view.getUint8(2),
      levelIndication: view.getUint8(3),

what about:

configurationVersion: reader.getNextUint8(),
      profileIndication: reader.getNextUint8(),
      profileCompatibility: reader.getNextUint8(),
      levelIndication: reader.getNextUint8(),

3. Parse udta box

In the context of the OSD Overlay tool, this box is pointless but basically there's a bunch of arbitrary metadata there that a writer can put in. Would be nice to parse it.

4. Generalise SampleEntry writer

Since we only ever write Avc1 boxes, all of the writing for this is bundled into one class. Really it's three boxes in one --- SampleEntry and then VisualSampleEntry. It would be good to have these broken out.