search

p-ranav / merged_depth

Monocular Depth Estimation - Weighted-average prediction from multiple pre-trained depth estimation models
MIT License
48 stars 3 forks source link

Relative depth to absolute #9

Closed semel1 closed 3 years ago

semel1 commented 3 years ago

In one of your response, related relative to absolute depth conversion, you mentioned : " I'm using some linear parameters to scale these to absolute depth". Could you please point to the particular code block where this transformation implemented. Thank you.

p-ranav commented 3 years ago

It's here. It's not great - actually doesn't seem to be a good way of doing it. That's because there is so much inconsistency between these networks. Adabins vs SGDepth vs MiDaS etc.. MiDaS does a great job separating out the features but using the other networks as a way to scale to absolute depth for MiDaS probably isn't the right way forward.

semel1 commented 3 years ago

Thanks for your prompt response. What would you suggest to perform such task (relative from MiDaS to absolute)?

p-ranav commented 3 years ago

There is no universal answer, I think. I don't have a good answer yet :(

The reality is that I'm getting different enough results from each of these networks that it really depends on your use-case (indoor vs outdoor for example) - if you pick a model that you're happy (e.g., some large but accurate resnet model), you could use such a model to scale up MiDaS and then use MiDaS as your absolute depth predictor.

Here's an example: Do you care about sky or not? Models based on KITTI are probably OK for autonomous driving but are pretty useless for any other usecase since it gets everything except road features completely wrong (sky for example). I wouldn't use that to scale MiDaS for a use-case that's predominantly indoor.

semel1 commented 3 years ago

Thank you