Add an example demonstrating media editing scenarios

ykh015 commented 4 years ago

One of the key use cases mentioned in the explainer is Non-realtime encoding/decoding/transcoding, such as for local file editing

Can we add an example demonstrating basic media editing operations like trim and concatenation ?

We are trying to understand on how to use WebCodecs to achieve the same functionality as MediaBlob to ensure that it meets developer needs.

tlecoz commented 3 years ago

Yes please ! Add a working example about it ! I'm trying to get it working for days and can't figure out how to save a canvas-animated-content as a mp4 file. I'm sure it's very simple, please show us a basic runnable demo.

chcunningham commented 3 years ago

We now have a few samples here https://w3c.github.io/webcodecs/samples/

Including one that creates a webm file after using WebCodecs to encode the camera stream https://w3c.github.io/webcodecs/samples/capture-to-file/capture-to-file.html

Closing, as I think the path to basic trim / concatenate functions is probably clear from the above. Roughly:

if all you need is to concatenate two files, you don't need webcodecs at all. you can do that with a JS muxer like in the demo above.
if you want to trim in the middle of a GOP and then concatenate, you may want to use WebCodecs to create a new GOP (starting with a new I-Frame) from the trimmed GOP. To do this, you would
- decode the original gop,
- discard (close()) the VideoFrames that are outside the trim window
- start encoding frames that are within the trim window
- add those encoded frames (chunks) at the concatenation boundary
- append the other (untrimmed) gops after that (no need to transcode).

Please reopen if more to discuss.

tlecoz commented 3 years ago

Hello ! Thank you for the link to the samples :)

My message has been written in october 2020 and at this time there was almost no information at all. My project almost work now but thanks anyway :)

Actually, there is still a problem, I searched a lot but didn't find a way to demux webm files that use alpha channel as I reported here but got no help at all... Any idea ?

https://github.com/jscodec/jswebm/issues/18

Also, there is no example at all showing how to do the opposite : mux the EncodedVideoChunk into a mp4 file. I know how to do it using ffmpeg but I have no idea how to do it in pure javascript and there is no example at all. I'm sorry to ask some help but most of webdevelopers (including myself) are not very familiar with mux / demux task .

Same thing for AudioEncoder / AudioDecoder : no example at all yet. In my project, I'm forced to convert my mp3 into mp4 in order to use mp4box to demux my audio track. I'm sure it's doable using a mp3 demuxer but it's not easy to get into it from a classical webdeveloper point of view. I also have no idea how to mux the EncodedAudioChunk into a mp3 file in pure javascript.

It's not as obvious as you may think it is (when I ask my question in october 2020 I didnt even know what could be a muxer / demuxer and I'm sure that 99% of webdevelopers never heard about that - now I know, but it was not obvious - )

henrywong commented 3 years ago

@tlecoz I totally agree with you!

henrywong commented 2 years ago

@chcunningham Can you please provide an example showing how to do the opposite : mux the EncodedVideoChunk into a mp4 file. I'm stuck with this problem now.

tlecoz commented 2 years ago

Same here (mp4, not webm, and with audio if possible)

Le mer. 4 août 2021 à 11:22, henrywong @.***> a écrit :

@chcunningham https://github.com/chcunningham Can you please provide an example showing how to do the opposite : mux the EncodedVideoChunk into a mp4 file. I'm stuck with this problem now.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/w3c/webcodecs/issues/62#issuecomment-892504837, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACCTMKF3V2U3YAXKDHIMELT3EBEFANCNFSM4NNSWFMA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

dalecurtis commented 2 years ago

You'll need to use something like https://github.com/videojs/mux.js or https://github.com/gpac/mp4box.js/ they both have code in their repositories showing how to generate an mp4 file. mux.js also offers a Slack channel for assistance: https://github.com/videojs/mux.js#talk-to-us

We'll try to put one together in the future, but it's not a priority at the moment.

tlecoz commented 2 years ago

Hello Dalecurtis, You are everywhere, thank you for your message ! :)

I searched multiple times inside mp4box.js & mux.js repositories looking for something like this but I'm so noob in muxing/demuxing that I didn't find it :'( Excuse me to ask this directly but can you give us the url of the html file in the "test" folder of mp4box that show how to create a mp4 file ? I suspected it was possible to do it using mp4box but as I said in a previous message, it's not as obvious as you may think it is. (for a simple web developer like me)

Have a nice day and thank you again for your work and your commitment

Le mer. 4 août 2021 à 19:29, Dale Curtis @.***> a écrit :

You'll need to use something like https://github.com/videojs/mux.js or https://github.com/gpac/mp4box.js/ they both have code in their repositories showing how to generate an mp4 file. mux.js also offers a Slack channel for assistance: https://github.com/videojs/mux.js#talk-to-us

We'll try to put one together in the future, but it's not a priority at the moment.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/w3c/webcodecs/issues/62#issuecomment-892839938, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACCTMJBKAAT55UNBRPKWOTT3F2JBANCNFSM4NNSWFMA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

dalecurtis commented 2 years ago

These are the ones I saw: https://github.com/videojs/mux.js/blob/main/test/mp4-generator.test.js https://github.com/gpac/mp4box.js/blob/master/test/qunit-iso-creation.js

tlecoz commented 2 years ago

I'm trully surprised that I didn't see the mp4-generator_test.js in mux.js because the name is obvious 😅 I'll try the mp4-box version.

Thank you a lot for your help and your time and your work :D

Le mer. 4 août 2021 à 19:54, Dale Curtis @.***> a écrit :

These are the ones I saw: https://github.com/videojs/mux.js/blob/main/test/mp4-generator.test.js https://github.com/gpac/mp4box.js/blob/master/test/qunit-iso-creation.js

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/w3c/webcodecs/issues/62#issuecomment-892855002, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACCTMNQVFRXMKSPYIUSJDLT3F5ERANCNFSM4NNSWFMA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

tlecoz commented 2 years ago

Hello !

I tryed to create a mp4 using mp4box. It almost work but there is a problem. (i was in holidays, that's why I didn't try before)

https://github.com/gpac/mp4box.js/issues/243

w3c / webcodecs

Add an example demonstrating media editing scenarios #62