search

prs-eth / Marigold

[CVPR 2024 - Oral, Best Paper Award Candidate] Marigold: Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation
https://marigoldmonodepth.github.io
Apache License 2.0
2.42k stars 132 forks source link

Is it possible to train the model by feeding it rgb images + 8-bit relative depth maps ? #97

Closed YacineDeghaies closed 3 months ago

YacineDeghaies commented 4 months ago

I want to train this model on a new dataset, but before that, I want to ensure that the model is suitable for my manually made grayscale depth maps.

One example of my depth maps:

shot_0002_dm_gt_s_0128

nandometzger commented 4 months ago

Hi there, thank you for the interest.

I would assume that it also works with your manual depth maps. There is a cheap sanitycheck to check the capabilities of the VAE. You can encode and decode your depth map, which should result in an reconstructed depthmap. Other then that, I think you should just try. Hint: you can resume training from the marigold checkpoint :)