This repository contains code and models for our paper:
Vision Transformers for Dense Prediction
René Ranftl, Alexey Bochkovskiy, Vladlen Koltun
1) Download the model weights and place them in the weights folder:
Monodepth:
Segmentation:
2) Set up dependencies:
```shell
pip install -r requirements.txt
```The code was tested with Python 3.7, PyTorch 1.8.0, OpenCV 4.5.1, and timm 0.4.5
1) Place one or more input images in the folder input.
2) Run a monocular depth estimation model:
```shell
python run_monodepth.py
```
Or run a semantic segmentation model:
```shell
python run_segmentation.py
```3) The results are written to the folder output_monodepth and output_semseg, respectively.
Use the flag -t to switch between different models. Possible options are dpt_hybrid (default) and dpt_large.
Additional models:
Run with
python run_monodepth -t [dpt_hybrid_kitti|dpt_hybrid_nyu] Hints on how to evaluate monodepth models can be found here: https://github.com/intel-isl/DPT/blob/main/EVALUATION.md
Please cite our papers if you use this code or any of the models.
@article{Ranftl2021,
author = {Ren\'{e} Ranftl and Alexey Bochkovskiy and Vladlen Koltun},
title = {Vision Transformers for Dense Prediction},
journal = {ArXiv preprint},
year = {2021},
}@article{Ranftl2020,
author = {Ren\'{e} Ranftl and Katrin Lasinger and David Hafner and Konrad Schindler and Vladlen Koltun},
title = {Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
year = {2020},
}Our work builds on and uses code from timm and PyTorch-Encoding. We'd like to thank the authors for making these libraries available.
MIT License