search

google / dynibar

Implementation of DynIBaR Neural Dynamic Image-Based Rendering (CVPR 2023)
https://dynibar.github.io/
Apache License 2.0
852 stars 40 forks source link

How to get an accurate colmap pose？Unable to accurately reproduce the kid-running poses #31

Open sjdfkjawe opened 1 year ago

sjdfkjawe commented 1 year ago

Hello, thank you so much for making your work open source.

I used colmap to reconstruct the kid-running data poses，But the reconstruction is not as accurate as the poses you released. It seems that after removing the little boy in yellow clothes, some of the images only contain the ground, and there is no rich texture. May I ask which colmap version you are using? and can you share your colmap reconstruction parameters with me.

Looking forward to your reply very much，Thank you.

zhengqili commented 1 year ago

Hi I believe I used the default parameters for running Colmap, where I use sequential matching with window size 15. Do you have visualization of your reconstruction somehow?

sjdfkjawe commented 1 year ago

Hi I believe I used the default parameters for running Colmap, where I use sequential matching with window size 15. Do you have visualization of your reconstruction somehow?

Thank you for your reply, I also use SequentialMatching, but poses around 40 frames are always not very accurate (only ground, no red car can be seen). May I ask which options you reset in the following parameters? dynibarQ

sjdfkjawe commented 1 year ago

Hi I believe I used the default parameters for running Colmap, where I use sequential matching with window size 15. Do you have visualization of your reconstruction somehow? dynibarQ2

I used the image of 1270x714 size and mask the kid in black to run colmap. It looks like the red camera is continuous, but I compared the specific xyz values and found that the results in the middle frames are not as accurate as your results, and the render quality is slightly affected after I replaced the poses in your release data with my colmap results.

sjdfkjawe commented 1 year ago

dynibarQ2

zhengqili commented 1 year ago

Hi, how do you replace your poses with released data? Do you retrain the video depth and dynibar models after you replace your camera poses?

sjdfkjawe commented 1 year ago

Hi, how do you replace your poses with released data? Do you retrain the video depth and dynibar models after you replace your camera poses?

Yes, I did ablation experiments only related to poses_bounds_cvd.npy and source_vv_poses.npy, and I rendered different moments from a fixed perspective. For kid running dataset, problems began to occur after 45k steps. I would like to ask if there are any Settings in the code that will cause the network to train in a bad direction at about 45k steps，I don't know if it's caused by learning rate or some decay setting.

zhengqili commented 1 year ago

Have you checked the rendered images through source_vv_poses.npy are correct? And what problem you saw after 45K steps?