Confused about stereo matching code in unimatch.py

[JarvisLee0423]

Thanks for your excellent work! After saw your code in unimatch.py, I got a little bit confused about line 160 and the following warping part. I want to know why you warp the right feature when the disparity is not none? In my view, the main problem of stereo matching is to find the correct pixel position in right which can match to the pixel in left. This is why we have to use all pixels in left to compute correlation with all pixels in right, in order to find the best matching point. This is why the attention map has the natural structure to replace the computation of cost volume. But after warping, all the right feature will be warped to left space, which means the corresponding between left feature and warped right feature will be pixel by pixel without any offsets. Therefore, in this case, the correlation between left feature and warped right feature have to be element multiplication, not the type we used to compute attention map. However, in the following codes, you will use left feature and warped right feature to compute attention map which is not element multiplication. In my view, it seems to illegal. Can you tell me why you implement your model in this way?

[JarvisLee0423]

I had further reading your code. I thought that the reason why you implemented in this way is that after you have the first disparity map, all the following prediction is to generate the delta-disparity, therefore, you only need to find the correlation between left feature and warped right feature which is more close to left space but not the exactly left space. Thus, the attention map can find the correlation between left feature and current warped right feature to generate delta-disparity to further optimizing the original disparity. Is that right?

[haofeixu]

Yes, right.