search

Vanilagy / mp4-muxer

MP4 multiplexer in pure TypeScript with support for WebCodecs API, video & audio.
https://vanilagy.github.io/mp4-muxer/demo
MIT License
419 stars 32 forks source link

Add support for multiple data tracks #69

Closed thandal closed 3 weeks ago

thandal commented 3 weeks ago

Data tracks can be used to store timestamped information. This is used by drones to embed sensor data, or by projects like FakeProof (https://www.edom-electric.com/fakeproof/details).

The API is to create a muxer, and then call addDataTrack as many times as desired, then call addDataChunkRaw(trackId, ...)

Since dataChunks are always written immediately, I don't think they impact 'fragmented' support, etc.

Vanilagy commented 3 weeks ago

Only fitting that PR number 69 is a banger like this one 😂 I'll check it out later.

Vanilagy commented 3 weeks ago

This is lovely, thank you for implementing this! This is quite a niche usecase, but a valid usecase nonetheless!

I think the implementation is not quite complete; there is additional handling necessary for handle the fragmented case, which also complicates the code quite a bit. The major thing to pay attention to with fragmented files is that all chunks are written a monotonically sorted way, which means that all tracks need to wait for all other tracks before they can write themselves. This crosstalk is currently hardcoded with the one video and audio track, but this PR would introduce n more tracks.

I am planning to start work on a new library soon which combines mp4-muxer and webm-muxer into one and will offer a new & improved API, with arbitrary track count support. This means you can also have multiple video & audio tracks, which is something your link mentions:

image

It wouldn't fit the API of this current library to implement many-track support for something so niche. Instead, I'll add support for metadata tracks in the upcoming library. Once I get to it, I'll use this PR as a reference when I get there!

Hope you can understand my reasoning. Feel free to use your own fork with this feature in the meantime.

thandal commented 3 weeks ago

Absolutely understood -- I also felt that this strained the current (quite simple and clean) API a bit. I'm just happy that you're planning to support multiple tracks in the near future. And yes, my current fork works for my needs for now!

On Tue, Sep 24, 2024 at 3:19 PM Vanilagy @.***> wrote:

Closed #69 https://github.com/Vanilagy/mp4-muxer/pull/69.

— Reply to this email directly, view it on GitHub https://github.com/Vanilagy/mp4-muxer/pull/69#event-14387106222, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAT6ESHO5M5TJKPLTVDD64DZYG3K7AVCNFSM6AAAAABOYNEWQ2VHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJUGM4DOMJQGYZDEMQ . You are receiving this because you authored the thread.Message ID: @.***>