search

Yahweasel / libav.js

This is a compilation of the libraries associated with handling audio and video in ffmpeg—libavformat, libavcodec, libavfilter, libavutil, libswresample, and libswscale—for emscripten, and thus the web.
288 stars 18 forks source link

Can we get complete codec string from demuxed stream? #11

Closed blenderskool closed 1 year ago

blenderskool commented 1 year ago

I'm trying to use this library with WebCodecs for decoding videos. I went through the examples in https://github.com/ennuicastr/libavjs-webcodecs-polyfill repo and while it got me to a good start, I couldn't figure out how to get codec string automatically from this library while using VideoDecoder from WebCodecs. The examples use a hardcoded codec string which works if the file is in that format, but may not work for user-selected video files in different codecs.

Demuxed stream from this library contains codec information like codec_id, codec_type, codecpar, but none of this can be passed to codec field in VideoDecoder.configure. A bit more digging led me to this page which contains an enum defining the ids of codec_id, but I don't know how these map to the codec string expected by VideoDecoder. Is there a way we can get the complete codec string directly from this library so that it can be used with WebCodecs?

At a higher level my use case is for user-selected video files that I can demux using libav.js and decode using WebCodecs without having to ask the user for the codec of the file. In any case, thanks for this awesome library 🙂!

Yahweasel commented 1 year ago

libav has avcodec_string, which I don't link in for some reason, which gets the string name of the codec. I can add that to the function list so that it's usable (I'll have to make a binding, since it's a somewhat annoying function).

That being said, the names that ffmpeg uses don't conform to any particular standard. ffmpeg predates the codec MIME strings that WebCodecs uses. So, for instance, ffmpeg's name for H.264 is "h264", not "avc" that (IIRC) WebCodecs uses. So, you'd still have to manually map it. Still, that's a much more feasible manual mapping than anything that libav.js currently provides, so it should be added!

blenderskool commented 1 year ago

Thanks for the clarity. I did encounter avcodec_string in the docs which I couldn't find in libav.js. It would be great if it can be added and also sent as a part of the streams returned after demuxing. Sad to see both are incompatible, but having the ffmpeg codec string for manual mapping should be a good start.

I could help in contributing a change for this, but I'm still quite new to ffmpeg and wasm.

Yahweasel commented 1 year ago

Seems the right way to do this is avcodec_get_name. avcodec_string requires an already-initialized de/encoder, which sort of defeats the purpose. I'll add that and a test for it. It's quite straightforward, but would take a lot of diving into libav.js's setup for somebody else to do, or a couple minutes for me ;)

Expect a release with this within the hour. It'll also have a test that demonstrates it.

Yahweasel commented 1 year ago

avcodec_get_name is in release 3.9.5.1. See tests/test-demuxing-ids-simple.js for a demonstration.

blenderskool commented 1 year ago

Wow, thanks for the quick update!