frame-stepping / random access seeking

[sigurdle]

Hi,

Just started using webcodecs in a frame / video editor. When seeking to a frame (when the video is paused). in order to make certain that I get that frame, I need to call flush() on the decoder (it seems). When I then next time start to seek again, I need to start from a keyframe. So if I'm e.g single-stepping through frames with an arrow key, for every time I go to the next frame I need to restart decoding from the closest previous keyframe (which could be hundreds of frames), this makes this pretty much unusable, unless I cache a lot of future videoframes myself.

I hope that an option to avoid this will be put in there (an option to flush output frames without needing to restart from a keyframe on the next frame decode)

Thank you

[sandersdan]

in order to make certain that I get that frame, I need to call flush() on the decoder (it seems)

This is generally true, be aware though that there may be cases where this doesn't produce identical output to continuing decoding with more frames.

this makes this pretty much unusable, unless I cache a lot of future videoframes myself.

This is the approach I would recommend. You'll need to determine the best memory vs performance tradeoff for your particular application.

FWIW, Chrome's <video> implementation decodes from the nearest keyframe for single-frame seeks.

I hope that an option to avoid this will be put in there (an option to flush output frames without needing to restart from a keyframe on the next frame decode)

I think this is currently out of scope for WebCodecs because it cannot be implemented reliably by all user agents.

There are cases where specific implementations could provide this feature, so it's possible that a future optional extension could expose it (eg. an in-decode-order API).