search

nianticlabs / simplerecon

[ECCV 2022] SimpleRecon: 3D Reconstruction Without 3D Convolutions
Other
1.28k stars 120 forks source link

Question about ScanNet Intricsics #18

Closed fengziyue closed 1 year ago

fengziyue commented 1 year ago

Hi @mohammed-amr :

Thank you for sharing this amazing work, during reading your code, I have some questions about the intrinsic of the ScanNet dataset:

(1), The intrinsic loaded by SimpleRecon is from sceneXXXX_XX/sceneXXXX_XX.txt which is directly downloaded from the ScanNet server. While the intrinsic of NeuralRecon / VoRTX is from sceneXXXX_XX/intrinsic/intrinsic_depth.txt, which is originally read from the sceneXXXX_XX.sens file. I found that in many scenes these intrinsic are not identical. Could you share any insights behind the difference/choice?

(2), The intrinsic for depth and color images looks different, SimpleRecon seems using the depth intrinsic to build the cost_volume from image features, I think I must be missing some details.

Any response will be greatly appreciated!

Sincerely, Ziyue Feng

mohammed-amr commented 1 year ago

Thanks for your interest!

Indeed. We're aware of this and are already working on a fix. All intrinsics are scaled. It's a tiny difference at quarter resolution in the cost volume, but we'd like for the cleanest implementation possible, so we'll post an update soon.

fengziyue commented 1 year ago

Thank you for the reply!

So do you know why there are two different versions of the intrinsic and which is supposed to use? And will the using of depth intrinsic instead of image intrinsic affect the simplerecon performance?

411066282 commented 1 year ago

@fengziyue Have you figured out your question here? How to define a picutre's intrinsic parameters? And do you kown how to get metadata infered in this paper without a ios device, and put it into this programm?

mohammed-amr commented 1 year ago

Hello,

This is now fixed. It is unclear which set of intrinsics are in the sceneXXXX_XX/sceneXXXX_XX.txt, but after multiple people raised the issue we looked into it. The ones in the intrinsics folder seem to be more correct, as both depth and color intrinsics there are identical up to scale. The repo now uses those and new models have been uploaded.

Both depth and color intrinsics there are identical up to scale in ScanNet, so it's fine to use either as long as we scale the correct way, which we are.

Thanks.