pals-ttic / sjc

Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation (CVPR 2023)
https://pals.ttic.edu/p/score-jacobian-chaining
Other
500 stars 15 forks source link

About bs=1 and the use of the Monte-Carlo estimate #15

Closed j0seo closed 1 year ago

j0seo commented 1 year ago

Hello! Thank you for sharing your great work.

In the paper, it says "An additional contribution of ours beyond DreamFusion [41] is our analysis of the effect that the OOD problem has when using a denoiser on rendered images (Claim 1), and the PAAS method to address it. For the variance reduction technique, namely the use of the Monte-Carlo estimate ˆ on Eq. (16), or − (in DreamFusion), vs. on Eq. (15), we observe comparable performance between the two methods empirically for 3D generation."

But in the code, it seems that only one noise is created (bs=1). If I understand correctly, to perturb multiple random noises and average them, the variable 'bs' should be more than one. I'm curious if I understand it well. Thanks!

w-hc commented 1 year ago

In practice setting it to a higher number incurs a lot more compute, cuz the bottleneck is the 2d model. It seemed more economical to do more 3d updates that way.

j0seo commented 1 year ago

Thanks for the quick response! Have you tried experimenting with increasing 'bs'? I'm curious if the quality improves even though more computation is needed.

w-hc commented 1 year ago

There doesn't seem to be a significant difference.

j0seo commented 1 year ago

Thank you!