WebCodec demo doesn't seem to work

[marklundin]

Hey Matt! Just tried this out and the WebCodec.html page doesn't seem to work correctly. No errors, but the final video output doesn't run and the downloaded file seems corrupt. Something going on with the muxer?

[mattdesl]

Are you in chrome canary with web codecs enabled in about config ? If so, perhaps they changed some API recently and I need to update.

[marklundin]

In chrome stable, but with flag enabled. it doesn’t seem to be throwing any errors in the encoding however, but I guess it could be an api change

[marklundin]

Just digging into this @mattdesl. This seems to be happening on Windows. OSX seems to be fine. still unsure if this is somehting to to with the WebCodec api or the muxing though.

[tlecoz]

I think (but may be totally wrong) that the issues come from an update of Webcodec that happened since chrome87.

In Chrome 86, VideoEncoder generated EncodedVideoChunk that was "based on" annex-b . In chrome 87, VideoEncoder became buggy and EncodedVideoChunk was , at the same time, "based on" annex-b and avcc.

The VideoEncoder has been fixed since chrome89 and ouput avcc by defaut. If you want to comeback to annex-b (it was my case in my project) , you just need to add an extra property on the config object you "send" to VideoEncoder.configure

Here is an example of config that works for VideoEncoder with annex-b ; { codec : 'avc1.64001E', width:1920, height:i1080, displayWidth:1920, displayHeight:1080, hardwareAcceleration:"deny", // or "allow" if you want to use gpu avc:{format:"annexb"} }

Hope it helps ;) And sorry if it's not related to the subject

[mattdesl]

Finally coming back around to this, the latest version should in fact support both annexb (default) or avc (though you have to specify { format: 'avc' } when creating the MP4 encoder).

Here's a test case: https://codepen.io/mattdesl/pen/LYWvmyp?editors=1010

^ This actually runs in latest Chrome now (I'm on 91.0.4472.106, no need for Canary). You still need to enable Experimental Web Platform Features in Chrome flags.

[ringcrl]

Finally coming back around to this, the latest version should in fact support both annexb (default) or avc (though you have to specify { format: 'avc' } when creating the MP4 encoder).

Here's a test case: https://codepen.io/mattdesl/pen/LYWvmyp?editors=1010

^ This actually runs in latest Chrome now (I'm on 91.0.4472.106, no need for Canary). You still need to enable Experimental Web Platform Features in Chrome flags.

this test case does not generate mp4 in chrome dev（ 95.0.4628.3）

[zhangbenber]

I met this problem in Chrome Beta (94.0.4606.41) too.

In my case, it was because the absence of data property in EncodedVideoChunk, which is internal according to the WebCodecs Spec. So I tried to copy it out using copyTo() and it worked.

See PR.

[ziyunfei]

I have the same problem.

[mattdesl]

I've just pushed a new build, let me know if 1.0.5 is working (also updated the demo https://codepen.io/mattdesl/pen/LYWvmyp?editors=1010).

Thanks!

[benjamind]

Can confirm this is not working under windows Chrome 96, with experimental web platform features enabled. Stepping through the frame data is being passed to mp4-wasm, but the resulting buffer after closing the encoder only has 152bytes in it.

[benjamind]

Did a little more digging today.

Can't seem to get annexb to work at all, but switching the example back to avc and fixing the decoderConfig opts which have changed got me a working avc recording!

diff --git a/src/extern-post.js b/src/extern-post.js
index a028d01..f2d0abe 100644
--- a/src/extern-post.js
+++ b/src/extern-post.js
@@ -155,17 +155,17 @@ export function createWebCodecsEncoderWithModule(MP4, opts = {}) {
   }

   function write_nal(uint8) {
-    const p = MP4._malloc(uint8.byteLength);
+    const p = MP4.create_buffer(uint8.byteLength);
     MP4.HEAPU8.set(uint8, p);
     MP4.mux_nal(mux, p, uint8.byteLength);
-    MP4._free(p);
+    MP4.free_buffer(p);
   }

   function writeAVC(chunk, opts) {
     let avccConfig = null;
-    if (opts.description) {
+    if (opts && opts.decoderConfig && opts.decoderConfig.description) {
       try {
-        avccConfig = parseAVCC(opts.description);
+        avccConfig = parseAVCC(opts.decoderConfig.description);
       } catch (err) {
         error(err);
         return;

[zhangbenber]

I tried the solution @benjamind gives above, and it finally works on Windows Chrome 96 (and also on MacOS with avc encoding).

However the file generated on Windows seems to have a wrong duration, which is exactly the double of normal one. I feed it to ffmpeg and see lot of warnings:

Error while decoding stream #0:0: Invalid data found when processing input
[h264 @ 0x7fb8fa00b000] no frame!
Error while decoding stream #0:0: Invalid data found when processing input
[h264 @ 0x7fb8fa00bc00] no frame!
Error while decoding stream #0:0: Invalid data found when processing input
[h264 @ 0x7fb8fa00c800] no frame!
...

I guess that it may populates malformed frames when parsing avcc data. This is the file I generated on Windows, which has a duration of 20s, and should be 10s instead (300fr / 30fps).

https://user-images.githubusercontent.com/7821581/144732849-4d47d3f7-7dc7-4e6b-9325-a0a646b18be2.mp4

[1ucay]

Hello, I tried in Chrome 98.0.4758.101 on Android.

DOMException: Encoder creation error. he @ mp4.js:1043 Kr @ mp4.js:1106 t.createWebCodecsEncoder @ mp4.js:21 start @ webcodecs.js:62 await in start (asynchronní) (anonymní) @ webcodecs.js:105

Uncaught (in promise) DOMException: Failed to execute 'encode' on 'VideoEncoder': Cannot call 'encode' on a closed codec.