w3c / webcodecs

WebCodecs is a flexible web API for encoding and decoding audio and video.
https://w3c.github.io/webcodecs/
Other
939 stars 132 forks source link

I would like to know whether the encoded data can carry the configuration information of its decoder. If so, can you give me an example? #154

Closed TTiamo closed 3 years ago

chcunningham commented 3 years ago

I'm not sure what's being asked. Are you trying to construct the decoder or configure it?

Here's a demo that does both: https://tguilbert-google.github.io/webcodecs/mp4/index.html

Relevant code snippets:

  async function onFrame(frame) {
     // Do something with frame...
  }

  let decoder = new VideoDecoder({
    output: onFrame,
    error: e => console.error(e),
  });

decoder.configure(config);
// config = {codec: "avc1.4d401e", codedHeight: 360, codedWidth: 640, description: Uint8Array(39)}
chcunningham commented 3 years ago

It is possible (perhaps even likely) that the encoded data (e.g. the AV1 bitstream) may describe enough of the stream properties such that you could derive a VideoDecoderConfig that could be used to configure the VideoDecoder. Is this what you're aiming to do?

How much can be derived and exactly how to do so would be specific to the codec. Can you say what codec you're trying to use?

This is not generally how we expect folks to fill out the config though. Ideally, the config info would be somehow stored outside of the bitstream (e.g. in the track metadata of a webm file).

chcunningham commented 3 years ago

Please re-open if you have followup questions.