Allow skipping frames

[andreclaudino]

How may I sleep frames while decoding?

I want to configure a custom sampling rate for loading frames. As I am creating a machine learning project, only a sample of the frames is enough to for inference.

How could I load, for example, one frame on each second, skipping decoding the frames that I don't want to use? Actually, following the current examples all frames are decided yet I use only some of them.

[gcanat]

I have a similar use case so I've implemented it. You can have a look here. Note: it does not rely on video-rs (at least not for this part of the code).

It goes roughly like this:

use::video_io::VideoReader;

// ... inside a function which returns a Result<>
let vr = VideoReader::new(
    filename.to_owned(),
    compression_factor,
    resize_shorter_side,
    threads,
    true,
)?;
vr.decode_video()

Where compression factor can be 0.25 for example if you want to take 25% of the frames evenly spaced, so basically one frame out of 4.

[gerwin3]

Because of the way codecs work, there is no way to "skip" frames. All frames must be decoded in order. There is one trick though: You can only decode the keyframes (because they contain all info). Just split the decoder, read frames and only feed keyframes. You can't choose your own interval though.

[gcanat]

Indeed you need to iterate through the packets of the stream and avcodec_receive_frame for all frames, but you dont have to run them all through the scaler neither to convert them to ndarray. If you only do this for, say 1 frame out of 2, or 1 frame out of 4, it can greatly speed things up. Just my 2 cents :)