search

nianticlabs / simplerecon

[ECCV 2022] SimpleRecon: 3D Reconstruction Without 3D Convolutions
Other
1.28k stars 120 forks source link

A question about mask in meta data #22

Closed gyes00205 closed 1 year ago

gyes00205 commented 1 year ago

Hi authors, Thanks for your amazing works. I have a question about mask in meta data. In line 82 of geometry_utils.py, you use torch.maximum to make depth value greater or equal to eps https://github.com/nianticlabs/simplerecon/blob/7de5b451e340f9a11c7fd67bd0c42204d0b009a9/utils/geometry_utils.py#L62-L85

And in line 622-623 of cost_volume.py, you let mask_b = depths > 0. But depth value of each pixel is greater than zero because of torch.maximum() in previous step, so the mask_b will all be True.
https://github.com/nianticlabs/simplerecon/blob/7de5b451e340f9a11c7fd67bd0c42204d0b009a9/modules/cost_volume.py#L618-L623

Then you concatenate the mask of each source image into meta data, but these mask are all True in every pixel of each source image. Aren't these mask value of meta data redundant? Or is there some wrong of mask computation?

mohammed-amr commented 1 year ago

Excellent catch! I'll look into this and push a fix along with bug fix for intrinsics.

Thanks!

mohammed-amr commented 1 year ago

Hello,

This is now fixed. We're not using OpenCV and Kornia style normalization for projection. Thanks a lot for pointing it out!

mohammed-amr commented 1 year ago

See Kornia:

https://kornia.readthedocs.io/en/latest/_modules/kornia/geometry/conversions.html#convert_points_from_homogeneous

And openCV:

https://github.com/opencv/opencv/pull/14411/files