search

princeton-vl / RAFT

BSD 3-Clause "New" or "Revised" License
3.21k stars 624 forks source link

Flow result is a mess when the adjcent two frames are almost the same(no action or very small action). #123

Open jucic opened 2 years ago

jucic commented 2 years ago

image look at the above picture, when the adjcent two frames are almost the same(no action or very small action), Flow result is a mess, I guess this is because there is no static scene in the training dataset of RAFT(Flyingchairs,Flyingthings,sintel and so no), so I made some dataset of static scene(optical flow always equal to 0 ) and finetuned RAFT after sintel stage of train_standard.sh，however, it doesn't imporve sufficiently(still comes to a mess in this condition sometimes), anyone knows why and how to solve this? Thanks in advance!

eyildiz-ugoe commented 2 years ago

It's exactly the same story for me. I have a scene in which only one object rotates slowly and I get these messy frames often.

One solution to this would be to train the network from the scratch using our own data. Does anyone have instructions for that?

UcefMountacer commented 2 years ago

yeah, same for my situation. The model output is a mess at some frames. The situation I'm doing it for involves a small object rotating, and a constant scene.

Here is a comparison between a good output and a messy one:

image image

ckcraig01 commented 2 years ago

Dear all, this is because of the normalization issue, this flow_norm can solve all your problem: https://github.com/tomrunia/OpticalFlow_Visualization/pull/7

jucic commented 2 years ago

Dear all, this is because of the normalization issue, this flow_norm can solve all your problem: tomrunia/OpticalFlow_Visualization#7

oh my god! Thanks a lot! I can't beleive this simple problem should have confused me so long time

hirominn commented 2 years ago

This is where the normalization takes place. I think you can display with fixed saturation by editing here.
https://github.com/princeton-vl/RAFT/blob/aac9dd54726caf2cf81d8661b07663e220c5586d/core/utils/flow_viz.py#L125-L131