search

mli0603 / stereo-transformer

Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformers. (ICCV 2021 Oral)
Apache License 2.0
660 stars 107 forks source link

Reproduce pre-trained results #35

Open VitorGuizilini-TRI opened 3 years ago

VitorGuizilini-TRI commented 3 years ago

Hi, how can I evaluate the pre-trained models to reproduce the KITTI 2015 numbers? Thank you!

mli0603 commented 3 years ago

Hi @VitorGuizilini-TRI The training and evaluation scripts are provided in the scripts folder.

VitorGuizilini-TRI commented 3 years ago

I'm using the kitti_toy_eval.sh script, but pointing to the KITTI 2015 dataset, with the 200 training images, and the kitti_finetuned_model.pth.tar. Out of the box it only evaluates on 10 images, giving these results:

Epoch 0, epe 0.6943, iou 0.9984, px error 0.0204

If I modify the dataloader to evaluate on all 200 images, I get these results:

Epoch 0, epe 0.4727, iou 0.9987, px error 0.0116

I'm assuming these are the numbers I should be getting, right?

Dataset 3px Error EPE
KITTI 2015 training 0.79 0.41