isl-org / DPT

Dense Prediction Transformers
MIT License
1.96k stars 254 forks source link

Unit of the Absolute Depth #63

Closed mohanhanmo closed 2 years ago

mohanhanmo commented 2 years ago

Thank you very much for your awesome work first. I am just wondering, what is the unit/metrics of the predicted absolute depth? Is it in meter? Thanks!

laihuaijing commented 2 years ago

Thank you very much for your awesome work first. I am just wondering, what is the unit/metrics of the predicted absolute depth? Is it in meter? Thanks!

Did you solve it? How should I get the actual distance from the estimated depth map? thanks !

if model_type == "dpt_large": net_w==net_h=384 model=DPTDepthModel( path=model_path, bachbone="vitl16_384", non_negative=True, enable_attention_hooks=False, )

dimasheva1 commented 2 years ago

Please, refer to #62. Hope it helps

laihuaijing commented 2 years ago

Please, refer to #62. Hope it helps

Thank you!

mohanhanmo commented 2 years ago

Thank you very much for your awesome work first. I am just wondering, what is the unit/metrics of the predicted absolute depth? Is it in meter? Thanks!

Did you solve it? How should I get the actual distance from the estimated depth map? thanks !

if model_type == "dpt_large": net_w==net_h=384 model=DPTDepthModel( path=model_path, bachbone="vitl16_384", non_negative=True, enable_attention_hooks=False, )

By setting the flag of "--absolute_depth", the output will be the absolute depth without normalization. But I am not sure about the definition of "absolute depth" compared to the normalized depth

mohanhanmo commented 2 years ago

Please, refer to #62. Hope it helps

Thank you very much for your reply! If the output is scale and shift invariant and isn't linked to any real distance, then what is the difference between the absolute depth and the depth after normalization? Both of them should be scale and shift invariant, why dont we keep only the normalized output? Thanks!!

matkovst commented 2 years ago

Please, refer to #62. Hope it helps

Thank you very much for your reply! If the output is scale and shift invariant and isn't linked to any real distance, then what is the difference between the absolute depth and the depth after normalization? Both of them should be scale and shift invariant, why dont we keep only the normalized output? Thanks!!

The difference is that the absolute depth map preserves the real ratio between different depths (to the model accuracy, though), while normalized depth does not. Knowing correct scale α and shift β, you can estimate the real depth in your unit system from the absolute depth map applying . On the other side, after applying normalization to your absolute depth map you lose the ability to estimate the real depth from this normalized map, because normalization is a non-linear operation. You may need to normalize depth in order to display it as a picture, like cv::normalize(D_abs, D_normalized, 0.0, 255.0, cv::NORM_MINMAX, CV_8U); cv::imshow("D_normalized", D_normalized);

mohanhanmo commented 2 years ago

Please, refer to #62. Hope it helps

Thank you very much for your reply! If the output is scale and shift invariant and isn't linked to any real distance, then what is the difference between the absolute depth and the depth after normalization? Both of them should be scale and shift invariant, why dont we keep only the normalized output? Thanks!!

The difference is that the absolute depth map preserves the real ratio between different depths (to the model accuracy, though), while normalized depth does not. Knowing correct scale α and shift β, you can estimate the real depth in your unit system from the absolute depth map applying . On the other side, after applying normalization to your absolute depth map you lose the ability to estimate the real depth from this normalized map, because normalization is a non-linear operation. You may need to normalize depth in order to display it as a picture, like cv::normalize(D_abs, D_normalized, 0.0, 255.0, cv::NORM_MINMAX, CV_8U); cv::imshow("D_normalized", D_normalized);

Thanks a lot for the explanation! That is very helpful. So it is saying that, even though we might not now the real distance based on absolute depth without scale and shift, but we can still compare the depth of 2 images by comparing their distance from the absolute foreground depth to the absolute background depth, right?

matkovst commented 2 years ago

Thanks a lot for the explanation! That is very helpful. So it is saying that, even though we might not now the real distance based on absolute depth without scale and shift, but we can still compare the depth of 2 images by comparing their distance from the absolute foreground depth to the absolute background depth, right?

I'm not sure about 2 (different) images. Different scenes may have different scales and shifts. Given an image of some scene, you can calculate absolute distance from one object (at the front) to another object (in the back) in that scene and that will be proportional to the real distance between these two objects. You can also calculate the real distance if you know the underlying scale and shift for this particular scene. When you move to another image of another scene, there will be the second scale and shift, more likely different from the first ones.