correlation 1d

[DeepDeep123]

Hi, Guo,

I just want to test the stereo model to produce the disparity. But it occurs error for the "correlation 1d". The error is "Segmentation fault" and no other information. Can you figure out the reason, or can you share you stereo disparity results to me?

Thanks!

[xy-guo]

Could you provide more details? Have you modified the code?

[DeepDeep123]

Yes, I have modified the code. I just want to produce the disparity on KITTI 2015 training dataset (200 iamges pairs) using your pre-trained stereo model. In this case , I remove all other unrelated losses. I have checked that it occurs error in the following step: "corr = F.leaky_relu(self.corr(conv2a, conv2b), 0.1)" in stereo_model.py. Therefore, I can make sure that the error locates at function correlation 1d.

[xy-guo]

How about the input size? Have you changed the parameters of Correlation1d(...)

[xy-guo]

The original images are recommended to be a multiple of 64. It is better not to change the parameters of Correlation1d(...), since the GPU code may have some unexpected bugs.

If you can not solve the bug, maybe you can replace the correlation with some Pytorch operations:

# left_feature [B, C, H, W]   # right_feature  [B, C, H, W]
B, C, H, W = left_feature.shape
maxdisp = 40
corr = left_feature.new_zeros([B, maxdisp + 1, H, W])
for i in range(maxdisp + 1):
    if i > 0 :
        corr[:, i, :, i:] = (left_feature[:, :, :, i:]*right_feature[:, :, :, :-i]).mean(dim=1)
    else:
        corr[:, i, :, :] = (left_feature*right_feature).mean(dim=1)
corr = corr.contiguous()

[DeepDeep123]

Hi, Guo,

Thanks for your code for correlation 1d. Now I can run the model. However, the procuced disparity seems not correct. If I use the correlation with Pytorch operations, can your pre-trained model still be used as usual？

[xy-guo]

I forgot the disparity order. I think maybe you can try the inverse order?

# left_feature [B, C, H, W]   # right_feature  [B, C, H, W]
B, C, H, W = left_feature.shape
maxdisp = 40
corr = left_feature.new_zeros([B, maxdisp + 1, H, W])
for i in range(maxdisp + 1):
    if i > 0 :
        corr[:, maxdisp - i, :, i:] = (left_feature[:, :, :, i:]*right_feature[:, :, :, :-i]).mean(dim=1)
    else:
        corr[:, maxdisp, :, :] = (left_feature*right_feature).mean(dim=1)

corr = corr.contiguous()

[DeepDeep123]

I can produce correct disparity now. Thank you very much!