search

HKUST-Aerial-Robotics / MVDepthNet

This repository provides PyTorch implementation for 3DV 2018 paper "MVDepthNet: real-time multiview depth estimation neural network"
GNU General Public License v3.0
309 stars 72 forks source link

Question about warping the image to construct the cost volume. #3

Closed bdseal closed 5 years ago

bdseal commented 5 years ago

Hello, thanks a ton for the code. I use my own data but get strange warped images. The left and right images are consecutive frames captured by a mobile phone. Camera intrinsic parameters and the relative camera pose are also known. But my warped images are not as “good” as your sample data. It’s only a small part of a image with rotation. I don’t quite understand the warping part in your GetVolume() function and I am not very familiar with stereo matching. Could you point me on where this problem is?

WANG-KX commented 5 years ago

Dear bdseal,

Thanks for your interest in our work! From your description, I do not know the exact problem. However, the following parts may worth checking: 1, Our method relies on multiview observation. It is quite different from monocular depth prediction (e.g. monodepth). To generate good depth maps, you may need good translation and overlap between frames. Translation means the distence between the optical center of the frames. In your case, "consecutive frames" maybe too close to each other. 2, The transform estimation should be accurate. I do not know how you get the relative camera pose and their accuracy. 3, Bugs. Maybe you can check this issue to make sure you feed the right data.

Lastly, if you are willing, you can send me some sample pairs. I do not guarantee that I can fix it for you, but I would like to try:)

Regards, Kaixuan Wang

bdseal commented 5 years ago

I have checked all those parts you suggested, but don’t get any promising results. Could you give me your Email so I can send some testing samples? Thanks for your help!

WANG-KX commented 5 years ago

Please send to kwangap at ust.hk