Vchitect / Latte

Latte: Latent Diffusion Transformer for Video Generation.
Apache License 2.0
1.45k stars 147 forks source link

Re-implementation err on ffs experiment #36

Open dummy702 opened 4 months ago

dummy702 commented 4 months ago

Good job, but I have some questions on ffs ckpt inference experiment. 1)I set "ckpt" in ffs.sh to the folder related to https://huggingface.co/maxin-cn/Latte/blob/main/ffs.pt", set "pretrained_model_path" to the folder related to https://huggingface.co/maxin-cn/Latte/tree/main/vae. But the performance of video generation is bad. Is there anything wrong with my process?

https://github.com/Vchitect/Latte/assets/161445238/1fcc2c49-402e-4977-9684-6547ac0e17bc

2)Besides, I edit the code in sample.py. If I keep the code "samples = vae.decode(samples / 0.18215).sample", I will get "Segmentation fault". Therefore, I replace the code with the following. Is there anything wrong with my process? 截屏2024-02-27 下午5 14 47

maxin-cn commented 4 months ago

Good job, but I have some questions on ffs ckpt inference experiment. 1)I set "ckpt" in ffs.sh to the folder related to https://huggingface.co/maxin-cn/Latte/blob/main/ffs.pt", set "pretrained_model_path" to the folder related to https://huggingface.co/maxin-cn/Latte/tree/main/vae. But the performance of video generation is bad. Is there anything wrong with my process?

sample.mp4 2)Besides, I edit the code in sample.py. If I keep the code "samples = vae.decode(samples / 0.18215).sample", I will get "Segmentation fault". Therefore, I replace the code with the following. Is there anything wrong with my process? 截屏2024-02-27 下午5 14 47

I just tested my code and didn't find anything unusual. For the second question, perhaps you could reinstall your environment? Thanks for your interest.

https://github.com/Vchitect/Latte/assets/38418898/bbe200ea-3e25-4835-b726-b3201601cf36