search

Yahweasel / libav.js

This is a compilation of the libraries associated with handling audio and video in ffmpeg—libavformat, libavcodec, libavfilter, libavutil, libswresample, and libswscale—for emscripten, and thus the web.
288 stars 18 forks source link

Random access to packets #32

Closed reinhrst closed 7 months ago

reinhrst commented 8 months ago

I would like to use libav.js as demuxer to be used with WebCodecs to make a video player (thanks for all the amazing work here, making my life a lot easier :)). The video files are drag-and-dropped into the webpage, and might be too large to fit in memory.

Making the system play a videofile from start to end is easy enough (using ReadAhead file, libav.ff_read_multi and https://github.com/Yahweasel/libavjs-webcodecs-bridge.

However it's unclear to me how I could support random access in this video file. I could get/scan all packets when the video file is received (this takes about 30 seconds for a 4GB file-- not ideal, but acceptable), I don't want to store the packets in memory.

It seems to me that I could either save the packets to e.g. Origin Private File System, in order to access them later, or I could for every random access start parsing the video from the start until the requested frame is found.

Am I missing something? Is there any way to get quickly to a particular packet?

reinhrst commented 8 months ago

Part of the issue seems to be that I thought only the functions in https://github.com/Yahweasel/libav.js/blob/master/docs/API.md are available; by now I realise that these functions are just javascript combining a bunch of lower level functions, which do seem to allow more granular access.

I will try to come up with a PoC and post it when I have something.

Yahweasel commented 8 months ago

What you want is to implement a block reader device that performs whichever query is needed on read. See https://github.com/Yahweasel/libav.js/blob/master/docs/API.md#mkblockreaderdev . Then, yes, API.md does not discuss the underlying libav API (see the first paragraph of API.md :) ), and seeking is not part of the high-level API. You want libav's seeking functions.

Some references you may want to look at are the demo (demo/demo.html), which can seek, and libavjs-webcodecs-bridge ( https://github.com/Yahweasel/libavjs-webcodecs-bridge ), which bridges libav.js and WebCodecs.

reinhrst commented 7 months ago

Thanks! I should be able to find my way from here; if I run into something, I'd be happy to make another issue.