is stereovoxelnet possible to be trained with unrectified raw left/right images

[ynjiun]

and the resulting model can be used by non-calibrated (non-rectified) stereo camera?

the reason for this question is: in the real world in mass production, stereo calibration is a costly process and no guarantee to result in a "good" calibration given each attempt. Plus even a stereo camera are well calibrated, but over time on the road due to vibration, the stereo camera soon to become "non-calibrated". So the real useful stereo model has to be robust enough to handle non-rectified stereo pair. My curiosity about your model is: would stereovoxelnet be a good candidate for non-rectified left/right raw images pair application? or it is still limited and constrained by epipolar line assumption? Please advise. Thank you very much for your insights.

[lhy0807]

Thanks for the comment and suggestions. We only tested StereoVoxelNet on the standard stereo dataset, which includes rectified images. Therefore, the current version could not work on unrectified images.

However, I think you bring up an interesting question. I guess as long as the model could find the correspondence along the epipolar lines, the depth information could be retrieved.

As a side note, during one experiment, I forget to rectify the image during training and testing, and the results seem not so bad. I don't have any data on this since this wasn't the focus of the paper.

[ynjiun]

@lhy0807

As a side note, during one experiment, I forget to rectify the image during training and testing, and the results seem not so bad. I don't have any data on this since this wasn't the focus of the paper.

This is an encouraging sign that your model might be robust enough to extend to non-calibrated stereo applications.

I am able to generate depth information from the Carla simulator for various driving scenes with multiple camera models (rectified or non-rectified with our choices). Would you be interested to work together to generalize your model to a non-epipolar line constrained model or in another term is "the model capable of learning an epipolar line by itself given a non-rectified stereo training set". What do you think? If you are interested, we can discuss the project or potential jointly paper publication opportunities over the email. Please let me know. Thanks so much for your insights!

[lhy0807]

I also think that’s an interesting thing to investigate. Feel free to shoot me an email (in paper) to discuss more details.