omerbt / TokenFlow

Official Pytorch Implementation for "TokenFlow: Consistent Diffusion Features for Consistent Video Editing" presenting "TokenFlow" (ICLR 2024)
https://diffusion-tokenflow.github.io
MIT License
1.52k stars 134 forks source link

Why add_noise before sample_loop #47

Open LiuRicky opened 2 months ago

LiuRicky commented 2 months ago

Thanks for your owesome work.

In run_tokenflow_pnp.py, why using "noisy_latents = self.scheduler.add_noise(self.latents, self.eps, self.scheduler.timesteps[0])" line257 before "edited_frames = self.sample_loop(noisy_latents, torch.arange(self.config["n_frames"]))" line 258.

It seems directly using "edited_frames = self.sample_loop(noisy_latents, torch.arange(self.config["n_frames"]))" is fine since it starts at timestamp 0. And the pnp-diffusers (https://github.com/MichalGeyer/pnp-diffusers, which this project may refer) just use "edited_img = self.sample_loop(self.eps)".