search

Zhendong-Wang / Prompt-Diffusion

Official PyTorch implementation of the paper "In-Context Learning Unlocked for Diffusion Models"
Apache License 2.0
364 stars 10 forks source link

Why the prompt-diffusion could generate the depth map? #5

Open Yi-Qi638 opened 1 year ago

Yi-Qi638 commented 1 year ago

Hi~ thanks for the excellent job. But I am confused about why the diffusion model could generate the depth map. From my limited knowledge, did the diffusion model have the depth map decoder? As the depth estimation is a regression problem, it is not an RGB Image, right?

Zhendong-Wang commented 1 year ago

Hi Wangyiqi, thanks for your interest! Yes, depth estimation could be formulated as a regression problem, but we reformulate it to be a generation problem and we found stable diffusion could works well on the cross-domain generation given enough information.

"did the diffusion model have the depth map decoder? .. it is not an RGB Image, right" diffusion model has RGB decoder and RGB includes gray depth images.