search

isl-org / MiDaS

Code for robust monocular depth estimation described in "Ranftl et. al., Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer, TPAMI 2022"
MIT License
4.56k stars 634 forks source link

Training code #43

Closed tarashakhurana closed 4 years ago

tarashakhurana commented 4 years ago

Do you plan to release your training code sometime in the future? It would be really helpful to advance the research on monocular depth estimation!

If not, can you explain how the Pareto optimatility is ensured during training? It seems like there will also have to be an undo step in the training pipeline such that whenever the Pareto optimum is reached and the next backpropagation update disturbs this state, this update will have to be reversed.

ranftlr commented 4 years ago

There are no plans to release the training code.

I'm not sure if I understand your question right. The algorithm that we are using is based on multiple gradient descent. As such it converges to a Pareto-stationary point, which means that there is no update if you are in a pareto-optimal point. However, in practice this is only approximately true, as we are working with stochastic approximations of the objectives. Did you have a look at the corresponding paper that covers the algorithm (https://arxiv.org/abs/1810.04650)?

Sankar-CV commented 4 years ago

Dear author's,

Thank you so much for such a great work and for sharing the trained model. The proposed MiDaS model works on diverse scenes with significant performance. This is one of the great contributions for the research community. I would like understand two points here:

(1) The depth map inferred from pre-trained MiDaS model is in the form of inverse depth, even after doing inversion I am not able to get absolute depth. What is the method to get absolute depth from the inverse depth?

(2) The quality of depth predicted from pre-trined MiDaS model per frame is really good but it is also inconsistent/jittery over a video sequence. It would be grateful if the training code is also accessible so that MiDaS model can be improved further.

vasavamsi commented 4 years ago

Dear Authors,

Can you please provide the train script for the model, as it would be really helpful for see the performance of the model on other datasets.

Regards,

ranftlr commented 4 years ago

@Sankar-CV See for example #42 for answers surrounding relative depth.

There are still no plans to release the training code.

ranftlr commented 4 years ago

Closing due to inactivity.