Pretrained depth model v2 with DPT Large

[zizou8686]

Dear authors, thank for the excellent work!

I've tried using your pretrained depth model (omnidata_dpt_depth_v2.ckpt) on my data however the results are different from ones gotten in the demo page. Do you use the DPT Large backbone in the demo model? If true, may you release this pretrained model as well?

Thanks!

[Ainaz99]

Hi @zizou8686 !

Sorry for the late reply! The version2 released pretrained models are the same as the demo page. You only need to do some post-processing on your model predictions. You can use the code provided here: https://github.com/EPFL-VILAB/omnidata/tree/main/omnidata_tools/torch#run-our-models-on-your-own-image.

Thanks!

[puyiwen]

你好@zizou8686 !

这么晚才回复很抱歉！ version2 发布的预训练模型与演示页面相同。 您只需要对模型预测进行一些后处理。 您可以使用此处提供的代码：https ://github.com/EPFL-VILAB/omnidata/tree/main/omnidata_tools/torch#run-our-models-on-your-own-image 。

谢谢！

Hi, I want to know how you got the depth zbuffer data? It is obtained by taking pictures with a depth camera or by inferencing a big trained model?Thank you very much!!

[alexsax]

Hi! The depth data is rasterized from meshes, which are reconstructed from depth cameras and Matterports structure-from-motion pipeline. So these are not raw depth sensor readings, but reconstructed from multiple views.

The depth model is then trained to estimate these rasterized depth values, (specifically a scaled and shifted disparity, using Midas loss).

[puyiwen]

你好！深度数据是从网格中光栅化的，网格是从深度相机和 Matterports 结构到运动的管道重建的。所以这些不是原始深度传感器读数，而是从多个视图重建的。

然后训练深度模型以估计这些光栅化的深度值（特别是使用 Midas 损失的缩放和移动视差）。

I think I got it, thank you very much again! Sorry to bother you again, I also have a question about the depth zbuffer, is depth ground truth absolute depth or relative depth? I want to mix taskonomy data with NYU_depth_v2, so I need to know about it. Looking forward to your reply.

[puyiwen]

Sorry to bother you, I want to know how can I use Midas to get absolute depth?  

浦一雯 @.***

 

------------------ 原始邮件 ------------------ 发件人: "Sasha @.>; 发送时间: 2022年9月24日(星期六) 凌晨2:50 收件人: @.>; 抄送: @.>; @.>; 主题: Re: [EPFL-VILAB/omnidata] Pretrained depth model v2 with DPT Large (Issue #9)

Hi! The depth data is rasterized from meshes, which are reconstructed from depth cameras and Matterports structure-from-motion pipeline. So these are not raw depth sensor readings, but reconstructed from multiple views.

The depth model is then trained to estimate these rasterized depth values, using Midas loss (which is actually scaled and shifted log-disparity).

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[puyiwen]

Hi! The depth data is rasterized from meshes, which are reconstructed from depth cameras and Matterports structure-from-motion pipeline. So these are not raw depth sensor readings, but reconstructed from multiple views.

The depth model is then trained to estimate these rasterized depth values, (specifically a scaled and shifted disparity, using Midas loss).

Hi,I run the demo.py of omnidata_tools, and I found that the values of the depth is [0-1], is the resulting depth map a parallax map? How do I get an absolute depth map in meters? Can you help me? Thank you very much again!!

[Twilight89]

Hi! The depth data is rasterized from meshes, which are reconstructed from depth cameras and Matterports structure-from-motion pipeline. So these are not raw depth sensor readings, but reconstructed from multiple views. The depth model is then trained to estimate these rasterized depth values, (specifically a scaled and shifted disparity, using Midas loss).

Hi,I run the demo.py of omnidata_tools, and I found that the values of the depth is [0-1], is the resulting depth map a parallax map? How do I get an absolute depth map in meters? Can you help me? Thank you very much again!!

Hi, did you solve this problem ?