search

prs-eth / Marigold

[CVPR 2024 - Oral, Best Paper Award Candidate] Marigold: Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation
https://marigoldmonodepth.github.io
Apache License 2.0
2.07k stars 106 forks source link

About Recovering the Depth with Metirc. #28

Open Magicboomliu opened 6 months ago

Magicboomliu commented 6 months ago

Thank you for your outstanding work! It is very impressive to deploy a diffusion pipeline into monocular depth estimation.

As stated in the paper, the model works for affine-invariance depth estimation, since the depth normalization is not revertible, I wonder if I want to recover the depth with metric, what can I do?

In other words, all the affine-invariance depth has a global scale or offset factor, according to the Eq3 of your paper, the d2 and d98 depth values from the given image, which is instance-independent I guess. Is there any method to recover the true depth with the assistance of extra information, like camera intrinsic or stereo images baseline?

markkua commented 1 month ago

This recent arxiv paper might be helpful to this question: https://arxiv.org/pdf/2405.15619