search

pals-ttic / sjc

Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation (CVPR 2023)
https://pals.ttic.edu/p/score-jacobian-chaining
Other
500 stars 15 forks source link

About OOD issues. #20

Closed lingtengqiu closed 1 year ago

lingtengqiu commented 1 year ago

In your paper, you said the render image X which is out of distribution of 2D stable-diffusion.

To address this issue, you add Gaussian noises into X to obtain X_bar set. Next you take X_bar set as the diffusion inputs. However, as you said X is out of distribution, does X_bar is not out of distribution of diffusion model?

In my opinion, X_bar = X+ sigma * N is also out of distribution. Hence the results of denoise(X_bar, sigma) is also incorrect. I am very confused about this issue.
Could you help me solve this problem

w-hc commented 1 year ago

you are right. only with large enough noise could (x + σ z) look not so OOD. Scores computed on (x + 0.01 z) for example is definitely ood, and the scores will be harmful.

lingtengqiu commented 1 year ago

Thanks for your reply. It helps me so much.