search

idiap / ESLAM

Apache License 2.0
204 stars 21 forks source link

depth mask for mapper color loss #4

Closed rwn17 closed 1 year ago

rwn17 commented 1 year ago

Hi @MohammadJohari ,

Thanks for your excellent work! I find that the depth mask is not used for the mapper color loss https://github.com/idiap/ESLAM/blob/main/src/Mapper.py#L362. Is it intended since I find the depth_mask for tracking color loss can make a lot of difference to the system? Thanks in advance.

Bests, Weining

shaoxiang777 commented 1 year ago

Hi @MohammadJohari ,

Thanks for your excellent work! I find that the depth mask is not used for the mapper color loss https://github.com/idiap/ESLAM/blob/main/src/Mapper.py#L362. Is it intended since I find the depth_mask for tracking color loss can make a lot of difference to the system? Thanks in advance.

Bests, Weining

I guess the RGB image is more accurate than depth image. So all sampled pixels in RGB images could be considered. But in depth image, some pixels are not valid with value of 0. It is highly relevant to SDF value, so depth mask is also implied on SDF loss.

MohammadJohari commented 1 year ago

Hello,

Thanks for your interest in our work! Yes, it is intended. The intuition is that during mapping, we reconstruct the scene, and accordingly, we update the representation via RGB information. However, during tracking, we will only exploit the most reliable information. That is why we use depth_mask to exclude pixels from contributing to tracking.