moviepy returns empty frames if frames are not accessed strictly sequentially

[pgunn]

Expected Behavior

See below - the handle to a video file should not start returning zeros. The video file we're using with this is here: https://caiman.flatironinstitute.org/~neuro/caiman_downloadables/msCam13.avi

We found this while spotting the error in pims, using the moviepy backend, and eventually tracked it down to moviepy when considering directly using moviepy to avoid what we believed was a bug in pims, instead to find that it's actually a bug in moviepy.

Oddly, accessing frame 0 of the movie fixes the handle to the movie

Actual Behavior

mph = moviepy.editor.VideoFileClip(fn) mph.size [752, 480] mph.duration 50.0 mph.fps 20.0 np.array(mph.get_frame(0)).max() 173 np.array(mph.get_frame(2)).max() 206 np.array(mph.get_frame(1)).max() 0 np.array(mph.get_frame(0)).max() 173 np.array(mph.get_frame(1)).max() 188

Steps to Reproduce the Problem

Specifications

• Python Version: 3.11.5
• MoviePy Version: 1.0.3
• Platform Name: Linux (tested on Windows too)

[pgunn]

Following it down into the backend:

mph = moviepy.video.io.ffmpeg_reader.FFMPEG_VideoReader(fn)
>>> mph.get_frame(0).max()
173
>>> mph.get_frame(1).max()
188
>>> mph.get_frame(2).max()
206
>>> mph.get_frame(1).max()
0
>>> mph.get_frame(0).max()
173
>>> mph.get_frame(1).max()
188

I'm looking at the logic here: https://github.com/Zulko/moviepy/blob/master/moviepy/video/io/ffmpeg_reader.py#L218

But I find the existence of self.last_read and self.lastread as distinct variables to be puzzling. I'm guessing the bug is somewhere around there.

[pgunn]

It looks like, at least with our datafiles, any use of -ss before -i in the call to ffmpeg causes ffmpeg itself to return empty frames, while the second call to -ss (while not performant) returns data correctly.

Found this by modifying moviepy/video/io/ffmpeg_reader.py to print out the commandline its building with various calls, and looking over the if start_time != 0 block in initialize().

Something like:

ffmpeg -ss 10 -i /home/pgunn/caiman_data/example_movies/msCam13.avi -ss 10 -loglevel info -vf scale=752:480 -sws_flags bicubic -pix_fmt rgb24 /tmp/foo_noprelinear.avi

(really providing anything at all, float or integer, with a first -ss, gives us a blank movie.

Without that initial -ss, we get movies that work.

I'm wondering if it might be better to remove the first -ss from these calls, or at least provide an option to.

[pgunn]

I will follow up with the ffmpeg developers.

[pgunn]

https://trac.ffmpeg.org/ticket/10602