How to guarantee that refined depths have the exact same value at those valid pixels in sparse depth map?

XinJCheng / CSPN

Convolutional Spatial Propagation Network

498 stars 93 forks source link

How to guarantee that refined depths have the exact same value at those valid pixels in sparse depth map? #29

Open mqchen1993 opened 4 years ago

mqchen1993 commented 4 years ago

raw_depth_input = blur_depth ... if sparse_depth is not None: result_depth = (1 - sparse_mask) result_depth + sparse_mask raw_depth_input

你好，我有一个疑惑。为了保留sparse depth上的有效值，为什么不是用sparse上的值替换，而是用blur_depth。blur_depth是网络预测的，这个不一定准确啊？请麻烦帮我解答一下，谢谢！

mqchen1993 commented 4 years ago

@XinJCheng

UltronAI commented 4 years ago

I have the same confusion. And I have tried to use result_depth = (1 - sparse_mask) * result_depth + sparse_mask * sparse_depth which is as your description in your paper (or it's a mistake?), but in this case, the model can learn nothing.

And it seems that the sparse_depth is only used to compute this sparse_mask, and the model will complete the depth map based on the estimated coarse depth (raw_depth_input in your codes). Why does it work well for depth completion? @XinJCheng

sHaol0w commented 4 years ago

@UltronAI @mqchen1993 I think it's pointed out in the paper sec 3.3.1 clearly with the exact update equation and explanation ? The sparse_depth is embedded to a hidden space(blur_depth [at sparse_mask position] ).