About generalization experiments on Middlebury and ETH3D.

[Zhaohuai-L]

Hi,thank you very much for your great work.
I test the generalization performance of the checkpoint you provided on Middlebury and ETH3D, but the results are different from Table7. Is it because the checkpoint provided is training without data augmentation ? So i would like to know if i misunderstood or misused? Thanks.

The checkpoint for test is sceneflow.pth

The results on Middlebury(half) training sets is : All: 'EPE': [7.572738313674927], 'D1': [0.27418503363927205], 'Thres1': [0.4556864986817042], 'Thres2': [0.3513654222091039], 'Thres3': [0.3013058652480443] Noc: avg_test_scalars_nonocc {'EPE': [6.537938332557678], 'D1': [0.2406569098432859], 'Thres1': [0.41864077051480614], 'Thres2': [0.3155737062295278], 'Thres3': [0.2668188696106275]}

The results on ETH3D training sets is : All: 'EPE': [0.9156075097896434], 'D1': [0.06998444140157921], 'Thres1': [0.18718870177313132], 'Thres2': [0.10728459610362295], 'Thres3': [0.06998444140157921] Noc: 'EPE': [0.9049188627137078], 'D1': [0.06913529045548919], 'Thres1': [0.1841263070150658], 'Thres2': [0.10554461269768783], 'Thres3': [0.06913529045548919]

The results in Table7 is :

Some qualitative results about the checkpoint:

BTW,I am also curious about how the results of RAFT-Stereo in Table 7 are obtained?

The generalization result in the RAFT-Stereo paper is：

The results in Graftnet are the same as RAFT-Stereo:

[gangweiX]

Thank you for your attention. You can run evaluate_stereo.py to test generalization on Middlebury and ETH3D. We use 15 training pairs of Middlebury 2014 and 27 training pairs of ETH3D to test generalization.

[Zhaohuai-L]

Thank you for your reply. I reprodueced the results using that code, such an amazing work.

[jiayuzhang128]

such an amazing work!

[killwy]

I am also curious about how the results of RAFT-Stereo in Table 7 are obtained? There are some differences between your paper and RAFT-Stereo raw paper.

[gangweiX]

Thank you for your attention, our evaluation method is different from RAFT-Stereo in Table 7, please refer to our code, we evaluate the non-occluded regions.

[killwy]

Thanks for your response!