search

jzhangbs / Vis-MVSNet

Visibility-aware Multi-view Stereo Network
MIT License
235 stars 27 forks source link

Combining monocular depth estimation with multi-view stereo #23

Open CanCanZeng opened 3 years ago

CanCanZeng commented 3 years ago

Hi, thank you for sharing this great work! It's the best algorithm in MVS field I have seen.

I always wondering if we can combining monocular depth estimation with multi-view stereo, since sometimes, monocular depth estimation can handle structured and textureless situation, but multi-view stereo is more robust and scale consistent. So, is there any way to combine this two kinds of work?

And I recently read a novel work in monocular depth estimation, https://github.com/microsoft/S2R-DepthNet, they try to extract structure information to predict depth, it seem very promising, is it possible to train vis-mvsnet on the "depth-specific representation"?

jzhangbs commented 3 years ago

Hi, thanks for providing this new work.

I have quickly go through the paper. It seems that the structure map is used to get rid of RGB information that is irrelevant to depth estimation.

I am not familiar with monocular depth estimation. To me, the monocular depth estimation techniques can be helpful for RGBD depth completion or filling in missing depth values in textureless regions.

If you use structure maps as inputs to MVS, I guess the result will be bad because the structure maps seems to be textureless.