search

nianticlabs / manydepth

[CVPR 2021] Self-supervised depth estimation from short sequences
Other
621 stars 84 forks source link

Question of relative pose in matching augmentation. #61

Open JarvisLee0423 opened 2 years ago

JarvisLee0423 commented 2 years ago

Hi, after I look through the codes of the many depth, I found a confused part when doing the matching augmentation. When you try to solve the static camera problem, you replace the frame -1 to be the color augmented version of frame 0. However, in the code, it seems that you only change the rgb from frame -1 to frame 0, the pose is still the relative pose between frame -1 and frame 0. Therefore this original relative pose will be used to compute the cost volume. Whereas, I think it is a little bit unreasonable, because in this case, when you compute the cost volume, the two frame will be augmented frame 0 and frame 0, whose relative pose should be identity matrix, but, in practical, the corresponding relative pose is between frame -1 and frame 0. It is confused for me. Looking forward to your explain.

DavidYan2001 commented 3 months ago

Hi, I find the same question, have you solved this? Btw, I suppose that the static camera aug and missing cost volume aug can be combined as one aug because we only need to set the pose=0, and input frame 0 and frame 0