Training / Evaluation instruction

[imneonizer]

Kudos to the authors! This repository is one of a kind, after looking through various depth estimation models on Github all of them have a similar problem i.,e with the perspective change they perform poorly and their depth estimation cannot be scaled back to real-world metrics.

Your repository seems to address those two big problems. I wanted to test out the trained models as provided before moving forward to training on my own dataset. However, there are just too many models which I don't understand how to use to get the final results i.,e real-world metrics.

This is what I have understood so far about model training process:

Let's say I want to train on KITTI dataset and then later adapt the trained model to oxford robot car dataset.

1. An unscaled depth network is required, which can be trained using our training procedure without using the scale recovery losses.

   python3 run_mono_training.py \
   --img_resolution 'med' \
   --flow_type 'classical' \
   --camera_height 1.70 \
   --stereo_baseline 0.52 \
   --pretrained_plane_dir 'results/plane-model-kitti-202101072240' \
   --data_dir '/media/datasets/KITTI-odometry-downsized-stereo' \
   --estimator 'orbslam' \
   --estimator_type 'mono' \
   --train_seq '00_02' '02_02' '06_02' '07_02' '08_02' '00_03' '02_03' '06_03' '07_03' '08_03' '11_02' '11_03' '13_02' '13_03' '14_02' '14_03' '15_02' '15_03' '16_02' '16_03' '19_02' '19_03' \
   --val_seq '05_02' \
   --test_seq '09_02' \
   --date $d \
   --lr 1e-4 \
   --wd 0 \
   --num_epochs 25 \
   --lr_decay_epoch 4 \
   --save_results

   • Where does the pretrained_plane_dir i.e., results/plane-model-kitti-202101072240 come from ?
   • Does it already contains some model files or is it a blank directory where some model file will be generated by this script ?
   • Where does the trained model is saved ?

2. Use the unscaled depth predictions from this network to train the ground plane segmentation network

   python3 run_plane_training.py  \
   --pretrained_dir 'results/final_models/vo-kitti-unscaled-202102201302' \
   --data_dir '/media/datasets/KITTI-eigen-split' \
   --date $d \
   --lr 1e-4 \
   --num_epochs 10 \
   --lr_decay_epoch 6 \
   --save_results

   • Again where does this pretrained_dir i.e., results/final_models/vo-kitti-unscaled-202102201302 come from ?
   • Does it already contains some model files or is it a blank directory where some model file will be generated by this script ?
   • What happened to the model trained in previous step ?
   • Where does the trained model is saved now ?

3. Our final model was chosen by selecting the one that resulted in the lowest validation loss

   • Which final model is being selected ?

4. The hyper parameters for the plane segmentation network training were λplane = 25, and λreg = 0.05. Fixing λplane, we reduced λreg until the plane segmentation masks were conservative enough to cover the majority of the road without erroneously producing high weights for off-plane pixels.

   • There is no instruction for training the final model

Please share a more detailed instruction on:

• Training / Testing the model
• Retraining to adapt on new dataset
• Inferencing on images to get the final real-world metrics