search

zju3dv / manhattan_sdf

Code for "Neural 3D Scene Reconstruction with the Manhattan-world Assumption" CVPR 2022 Oral
https://zju3dv.github.io/manhattan_sdf/
Other
491 stars 35 forks source link

Question about COLMAP generated depth map #11

Closed Cli98 closed 2 years ago

Cli98 commented 2 years ago

Hi Author,

Please allow me to show my appreciation for this great work!

I have a question regarding data preparation #5 , where you give a guideline. I followed your instruction on #7 to generate sparse and dense reconstruction following the colmap provided by NerfingMVS.

However, after obtaining predicted depth map by loading files under 'dense/stereo/depth_maps', image_name + '.geometric.bin' as in #7 and transform in numpy array, I compared both yours and my prediction. I found that there is a large difference. After I replace the colmap depth of my version and retrain, I can generally converge, but the appearance is not good. There are large area of distortion on floor.

How did I generated depth prediction?

  1. following your instruction to download colmap version as in NerfingMVS. The version support fusion ops. I manually verify this following issues in NerfingMVS.
  2. run script here https://github.com/weiyithu/NerfingMVS/blob/main/colmap.sh to generate depth map with sparse+depth reconstruction.
  3. gather depth map as the function your refer to in #7 and save them to numpy array

I have no idea why there is a large difference. So here is my question,

  1. Are you doing the similar as mine? Have you normalized the colmap depth result?
  2. Do we need to adjust pose according to generated colmap depth? I use your pose instead. Not sure if it is related. Actually I have no idea how to adjust pose :D.
  3. for question 1, if yes, can you instruct me the possible reason that I failed to generate good result? If no, can you tell me which part I missed?

Thank you and really appreciate if you can help answer my question.

ghy0324 commented 2 years ago

Hi! Thanks for your interest in our work!

I'm not sure what the problem is. We have just provided a detailed instruction here, including a Python script to run COLMAP. Please check it out first, and feel free to ask me if it's still not solved or you have further questions.

Cli98 commented 2 years ago

@ghy0324 Thank you for providing the script. I will test it and return back to you if I'm not able to solve the problem.