How to normalize the disparity (or depth) to feed into VAE in RichDreamer?

[thucz]

Hi! Since Monocular Depth Network provides a relative disparity. How do you normalize the disparity (or depth) to feed into VAE?

[lingtengqiu]

You can see the supplementary from arxiv version Instead of disparity which has very poor property, we use relative depth ( setting near and far plane) to feed into VAE

[thucz]

Thanks for your reply! But I still wonder what values you set the near and far planes for the predicted depth for Monocular Depth Network. Could you give me a script of how you exactly process the predicted disparity of MiDaS on LAION datasets?

[lingtengqiu]

For the predicted depth from Midas, we normalize disparity (dis - min(dis)) / (max(dis) - min(dis)), given a prompt "xxx". On the other hand, when we finetune multi-view normal-depth diffusion model on objaverse, we transfer disparity domain to inverse depth domain, given a prompt "xxxx, 3D asset."

[thucz]

Thanks! I got it.