Great work! A question about sampler_method. - Githubissues

PKU-YuanGroup / Open-Sora-Plan

This project aim to reproduce Sora (Open AI T2V model), we wish the open source community contribute to this project.

MIT License

11.55k stars 1.03k forks source link

Great work! A question about sampler_method. #294

Open jiaxiangc opened 5 months ago

jiaxiangc commented 5 months ago

When I choose DDIM as sampler method, the results are bad. The results from PNDM sampler method are great. I want to know the scheduler configs when using DDIM sampler method.

jiaxiangc commented 5 months ago

DDIM Inference results, very strange.

LinB203 commented 5 months ago

That's a good question we'd like to know too./(ㄒoㄒ)/~~

jiaxiangc commented 5 months ago

请问你们在训练的时候，Scheduler的设置是什么，有没有可能是训推没对齐？

LinB203 commented 5 months ago

It seem that it is a bug in diffusers. We should add clip_sample=False to DDIMScheduler and DDPMScheduler.

jiaxiangc commented 5 months ago

It seem that it is a bug in diffusers. We should add clip_sample=False to DDIMScheduler and DDPMScheduler.

好的，感谢哈，我尝试一下是不是这个问题。

jiaxiangc commented 5 months ago

我打印了decoder的sample范围，大概在-4，4. 如果clip_sample了，那么范围就在-1，1了。这个是不是因为你训练VAE的时候，没有预处理视频的原因？或者是 casual 3d vae 就是会导致这种情况？