search

jjihwan / SV3D-fine-tune

Fine-tuning code for SV3D
MIT License
91 stars 5 forks source link

What is the meaning of cond_aug #12

Closed 2hiTee closed 7 hours ago

2hiTee commented 4 days ago

Hi, thanks for your work! I have a question, why need to add noise to the first frame condition? As shown in the sv3d pipeline, it did not show that any noise should be added to the first frame condition. Below is the code. cond_sigmas = torch.Tensor([1e-5]) ## ToDo cond_frames = torch.rand_like(cond_frames_without_noise) * cond_sigmas + cond_frames_without_noise

jjihwan commented 7 hours ago

It is typical noise augmentation method proposed in Cascaded Diffusion Models for High Fidelity Image Generation. You can find the details in section 4.3 of SVD paper, since SV3D directly utilize SVD.