Inference code for arbitrary images

[vabrishami]

Hi, and thank you for your excellent work. Currently, there is no script to apply this model to arbitrary images (frames). Do you have any plans to provide one? Thanks, and congrats for acceptance!

[sgasperini]

Hi @vabrishami, thank you for your interest in our work!!

What do you mean by "arbitrary"? Do you mean any input image?

Please consider that ours are not foundation models. Our models were trained only on a single autonomous driving dataset (i.e., outdoors, driving scenes), so we do not expect them to deliver reasonable estimates with all inputs, e.g., indoors.

If you meant taking our pre-trained models and inferring on different outdoor driving images, for example, from KITTI or other datasets, our models might generalize better than the baselines, but we did not test that. It would mean creating a new dataset/dataloader for the new inputs, similar to those for RobotCar (data/robotcar/robotcar_dataset.py) or nuScenes (data/nuscenes_dataset.py). For such a transfer, it is recommended to adjust the image size of the new data to be similar to the one the model was trained for, preserving the aspect ratio, so this might involve center cropping (e.g., to use KITTI inputs, which are much wider). Alternatively, you can re-train our model on your data. This means training the GAN for image translation first (translating from easy inputs to hard ones), the depth baseline second (only on the easy ones), and the final depth model third (on the mix of easy and hard, distilling knowledge from the baseline which is always fed the easy ones).

Let us know if this answers your question. Thank you!

[vabrishami]

Hi @sgasperini. Yes, something like test_simple.py in the monodepth2 repo can be handy for rapid testing on how it works on other data. But I got your point, and thanks for your very complete explanations. Keep up the excellent work!

[sgasperini]

Hi @vabrishami, I see what you mean. We are currently overwhelmed with other things, but we may add that later. We will post an update here in case.

Thanks a lot!! Very appreciated

[morbi25]

Hi @vabrishami, we now added the test_simple.py to simplify quick tests. Nevertheless, please remember that our models were trained on a single dataset so we provide no performance guarantees on the transfer to out-of-distribution data (as @sgasperini already pointed out). If you face any problems while using the script, do not hesitate to reach out again or reopen the issue.

[vabrishami]

Thanks! Fantastic work!