search

gengshan-y / expansion

Upgrading Optical Flow to 3D Scene Flow through Optical Expansion, CVPR 2020 (Oral).
https://gengshan-y.github.io/expansion/
MIT License
172 stars 27 forks source link

Finetuning on my own dataset #16

Open Abhinav275 opened 3 years ago

Abhinav275 commented 3 years ago

Hi,

I tried to fine-tune the model on my own dataset. The code runs and finetunes the model. However, when I use the finetuned model with test images, the expansion and motion-in-depth seem to have quite noisy results.

I am using Habitat Simulation to generate my dataset. In the attached image, the camera is moving forward and the ball is falling on the ground. I was hoping that the ball could be more distinct in the expansion and motion-in-depth.

Is there something that I might be missing?

Thank you.

Original Image image

Optical Expansion image

Motion in Depth image

gengshan-y commented 3 years ago

Hi, the expansion results does not seem to give meaning information about the scene depth. I'm not sure if that is due to the way two frames are sampled. Are there enough expansion between the frames?

If you look at paper Sec. 4.5, the depth accurate is highly dependent on the magnitude of expansion. Therefore, a concrete suggestion is to compute expansion between two far-away frames, such that the expansion is large enough.