The optical flow between two identical images looks like random noise. Is this normal?

[JWargrave]

Hi, there. Thanks for your great work!

I am new to optical flow estimation, and I want to integrate optical flow into my own project as an out-of-the-box tool.

I run the script below on my own video, but I found that many frames in the optical flow video look like meaningless random noise.

# inference on video
# script from: https://github.com/autonomousvision/unimatch/blob/master/scripts/gmflow_demo.sh
CUDA_VISIBLE_DEVICES='1' python main_flow.py \
--inference_video video_path \
--resume pretrained/gmflow-scale2-regrefine6-mixdata-train320x576-4e7b215d.pth \
--output_path result_dir \
--padding_factor 32 \
--upsample_factor 4 \
--num_scales 2 \
--attn_splits_list 2 8 \
--corr_radius_list -1 4 \
--prop_radius_list -1 1 \
--reg_refine \
--num_reg_refine 6 \
--save_video #\
# --concat_flow_img

Then, I calculate the optical flow between two identical images and found that the result also seems like random noise. (Shouldn’t it be an all-white picture?) I guess the above problem may be caused by this, because some scenes in my video are static.

Because I know little about optical flow, I would like to ask if this phenomenon is normal? If not, how to solve it?

Thanks a lot.

[JWargrave]

Maybe the answer under this issue can explain this problem?

[haofeixu]

Hi I think this could be expected since the model has never seen this kind of training data and thus produces unreliable results.