When the video length is more than 1024 frames, only 1023 pairs of optical flow images can be extracted？

[zugexiaodui]

When I calculated the optical flow for a folder containing 1298 frames, I only got 1023 pairs of optical flow images. For other folders with more than 1024 frames, only 1023 pairs of optical flow images can be obtained at most.

I use os.system(cmd_line) in the python script, where cmd_line is: f"denseflow {video_dir_path} -b={dfparm_d} -a={dfparm_a} -s=1 -if -v". dfparm_d = 20 and dfparm_a='tvl1'. video_dir_path is the folder path of the frames.

For the folders which contain less than 1024 frames, I got all the optical flow images. However, when I calculated the optical flow for a folder containing 1298 frames, the output information is: push frames gray, video_flow_idx 0, batch_size 512 push frames gray, video_flow_idx 511, batch_size 512 push frames gray, video_flow_idx 1023, batch_size 275 loaded video "**path**/01_023", 1299 frames load frames exit. 1 videos (1299 frames, 1298 tvl1 flows) processed, using 28.739s, decoding speed 45.1999fps, flow speed 45.1651fps

In my video_dir_path, the file name of the frames are 0001.jpg ... 1298.jpg. In the output_dir(I have cd into it so that the optical flow images are output to it), the file names are flow_x_00000.jpg...flow_x_01022.jpg, and flow_y_00000.jpg...flow_y_01022.jpg. Also, the output information (1299 frames, 1298 tvl1 flows) are not consistent with the frames and flows. Is that a bug or my fault? Thanks!

[innerlee]

you can also try to extract frames using denseflow, by setting -s=0

[innerlee]

videos are complex. Different decoder may output different number of frames.

[innerlee]

Sorry didn't read your post carefully. Could you please post the result of ls THAT_VIDEO_FOLDER?

[zugexiaodui]

Thank you for your reply!

My script: extract_opticalflow_2.txt

My video folder (ls 01_023): src_video_folder.txt

Output: When I set -s=0: denseflow_s0.txt (The frames are only copied.) -s=1: denseflow_s1.txt

[innerlee]

This is strange. I will check if there is a bug in writing flow filenames

[innerlee]

Could you please turn on verbose option -v when extracting flow, and post the output log?

[zugexiaodui]

The output log:

(torch17) luyue@gpu-Super-Server:~/workspace/Projects/MMDetSeg_2/OpticalFlow$ python extract_opticalflow_2.py 
"/home/luyue/workspace/Projects/MMDetSeg_2/data/SHTCampus/Train/01_023", frames ≈ 1298
push frames gray, video_flow_idx 0, batch_size 512
push frames gray, video_flow_idx 511, batch_size 512
push frames gray, video_flow_idx 1023, batch_size 275
loaded video "/home/luyue/workspace/Projects/MMDetSeg_2/data/SHTCampus/Train/01_023", 1299 frames
load frames exit.
1 videos (1299 frames, 1298 tvl1 flows) processed, using 28.586s, decoding speed 45.4418fps, flow speed 45.4068fps
Finished!

[innerlee]

hmm, nothing wrong.

could you pls reduce the number of frames into 600, and check if the remaining 80 are lost?

[zugexiaodui]

I still got 600 pairs of flow images. ls video_dir: src_600_01023.txt ls flow_dir: flow_600_01023.txt

[innerlee]

Try to make a list of 2000 frames, and see if it fails on the 3rd batch

[zugexiaodui]

I find that the frames are not clear so I extract the frames of the raw video again. The old frames are extracted by ffmpeg command. The new frames are extracted by opencv and I set the IMWRITE_JPEG_QUALITY=100. Old frames have been deleted.

I extract optical flow images for the new frames, and it doesn't fails.

"/home/luyue/workspace/Projects/MMDetSeg_2/data/SHTCampus/Train/2000_01023", frames ≈ 2000
push frames gray, video_flow_idx 0, batch_size 512
push frames gray, video_flow_idx 511, batch_size 512
push frames gray, video_flow_idx 1023, batch_size 512
push frames gray, video_flow_idx 1535, batch_size 465
loaded video "/home/luyue/workspace/Projects/MMDetSeg_2/data/SHTCampus/Train/2000_01023", 2001 frames
load frames exit.
1 videos (2001 frames, 2000 tvl1 flows) processed, using 49.935s, decoding speed 40.0721fps, flow speed 40.0521fps

I checked the output dir and got 2000 pairs of flow images. Then I extracted optical flow for the original 1298 frames video. I got 1298 pairs of flow images. It seems correct... I cannot repeat my 'bug' now. I don't know why. However, denseflow works. Thanks for your help.

[innerlee]

Actually you can extract frames using denseflow, with the -s=0 option, as mentioned before.

Anyway, good to hear that the problem has gone.

[zugexiaodui]

I'm sorry for that I misunderstood what you said about -s=0. Thanks again.

[ChuhuaW]

Hi, I encounter a very similar issue. I have 11588 jpg frames in my folder, but only got 11263 pairs of flow. Tried extract frames by setting -s=0, and run denseflow again but it didn't help. I have other folders with more than 20000 frames and it works fine with denseflow.

[ChuhuaW]

denseflow_s1.txt Please see the log attached. Seems like the last batch was not saved.

[innerlee]

Okay the log says 1 videos (11588 frames, 11587 tvl1 flows) processed

[ChuhuaW]

Okay the log says 1 videos (11588 frames, 11587 tvl1 flows) processed

Thanks for the reply. I'm attaching the screenshot showing file counts as well. Only 11263 pairs of flow are saved.

[innerlee]

Is it possible to share the video file?

[ChuhuaW]

Is it possible to share the video file?

Absolutely! I use ffmpeg to extract the frames in 24 fps. ffmpeg -i test/video_test_0000007.mp4 -r 24 ./0007/%07d.jpg https://user-images.githubusercontent.com/16619342/151222045-388f03b9-eafa-42de-8edf-d375c4565f38.mp4