search

w3c / mediacapture-transform

MediaStreamTrack Insertable Media Processing using Streams
https://w3c.github.io/mediacapture-transform/
Other
44 stars 20 forks source link

Memory locality #30

Open aboba opened 3 years ago

aboba commented 3 years ago

@padenot is proposing a change to the WebCodecs API to improve memory locality. I would like to understand if this has any implications for Mediacapture-transform.

Currently, VideoFrame buffers can be released by calling close(), which will typically happen after they have been used as input (e.g. in WebCodecs encoder). Once released, VideoFrame buffers can presumably be reused, at the implementation's discretion. It therefore isn't clear to me whether there a way for mediacapture-transform to make use of the additional WebCodecs arguments that are being proposed. For example, would a BYOB reader be helpful?

chcunningham commented 3 years ago

Copying a bit of my comment from the WC bug:

Re: capture, this too uses pooling internally. See here for an overview of the IPC API that dishes out buffers and to observers (render processes) which then release them at some later point.

To the core question raised, I'm not sure yet. Waiting to see some rough psuedo code.