search

pals-ttic / sjc

Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation (CVPR 2023)
https://pals.ttic.edu/p/score-jacobian-chaining
Other
504 stars 15 forks source link

Why the gradient is divided by sigma? #6

Closed MultiPath closed 1 year ago

MultiPath commented 1 year ago

Based on the paper equation, it should divide by sigma^2 instead of sigma, right? https://github.com/pals-ttic/sjc/blob/main/run_sjc.py#L141

image
w-hc commented 1 year ago

I am on my phone. Will expand once I get to laptop.

If u check out the appendix, or Karras elucidating diffusion, the instantaneous update is $- \sigma \cdot score$. Also referred to as Twedie’s formula. Although here in case of PAAS, since both $D$ and $x$ are clean images, I’d like to believe you can even not devide anything. It’s really the l2 loss gradient between 2 images both with numerical range in [-1, 1]

MultiPath commented 1 year ago

I see. Thank you so much for the explanation!