hmorimitsu
commented
2 months ago Hi, thank you for your interest in our work.
Does the odd outputs mean that the flow visualization show some random colors? If so, that is probably being caused by the estimation noise. Optical flow models produce a low-magnitude noise in the estimation. When the scene contains motion, the noise is not apparent in the visualization because the magnitude of the motion is much larger than the noise.
The reason you can sometimes see the noise is because the flow visualization is first normalized to set the maximum motion magnitude of the scene as 1.0. Therefore, when there is no motion, the noise magnitude dominates the normalization term and become apparent in the visualization.
If you want to avoid visualizing low-magnitude noises, you can manually set a higher value for the flow_max_radius argument when using the flow_to_rgb function.
I hope this helps. If not, just let me know.
Best,
nauti16
Thank you for sharing your excellent work. Recently, I tested your model and made an interesting observation. I ran the model's inference using two consecutive frames, frame T and T+1, with almost no motion between them. However, the model produced very odd optical flow outputs, while it generated normal optical flow when there was any motion area in the images.
I'm curious about the reason behind this strange behavior. Is there something I might have missed? I would appreciate your insights on this behavior as I'm interested in better understanding the model's workings and uncovering potential limitations.