occlusion detection

[gpuartifact]

Hi, RAFT-Stereo is producing disparities also in regions that are occluded to one of the cameras of the stereo pair. As a result the disparites are good when scene is similar to the training set. In terms of generalization it then produces more errors in those partially occluded regions that are less similar to data in the training set.

In SGM implementations the left-rigth and right-left consitency check is used to implicitly find occluded regions. This check is done in SGM within the cost cube computed within one run (only left-right or only right-left) by swithching the Cost look up direction.

Where in the RAFT-Stereo implementation would you suggest implementing an equivalent for occlusion check without having to implement the full disparity computation twice? Thanks.

[lahavlipson]

Sorry for the late response.

The context features could be reused to compute flow in the opposite direction (although you might need to re-train). So flipping the cost volume and running additional GRU iterations to estimate right-left disparity as well should be relatively inexpensive, which you can use for a self-consistency check.

[TingmanYan]

The context features could be reused to compute flow in the opposite direction (although you might need to re-train). So flipping the cost volume and running additional GRU iterations to estimate right-left disparity as well should be relatively inexpensive, which you can use for a self-consistency check.

Hi, I swapped the left and right images and fed them into RAFT-Stereo to compute the right-to-left disparity. However, the result is not as accurate as the left-to-right disparity: Does this mean that to compute the right-to-left disparity, re-train is necessary? Or am I missing anything to compute an accurate right-to-left disparity?