modelscope / DiffSynth-Studio

Enjoy the magic of Diffusion models!
Apache License 2.0
6.59k stars 600 forks source link

Diffutoon 是否可以生成多种风格不同的视频? #206

Open LesenlyMu opened 2 months ago

LesenlyMu commented 2 months ago

我想要生成更多风格的动漫视频,应该在什么地方进行修改。我尝试过修改 stable diffusion 模型,但似乎没有明显的变化。

wenxianxin2000 commented 2 months ago

修改提示词,提示词很重要

Artiprocher commented 1 month ago

Diffutoon 是只考虑动漫风格视频的渲染方案,如果想要生成其他风格的视频,建议把 ControlNet 模型修改成 Depth 和 SoftEdge,并使用其他 Stable Diffusion 底模,另外分辨率需要适当降低。

nitinmukesh commented 1 month ago

Diffutoon is a rendering solution that only considers anime-style videos. If you want to generate videos of other styles, it is recommended to modify the ControlNet model to Depth and SoftEdge, and use other Stable Diffusion base models. In addition, the resolution needs to be appropriately reduced.

Thanks for the tip. I tried this combination before (Depth and SoftEdge), for some reason it take large amount of time. I tried this again today for 1s / 30 frames / 30 fps / W:576 / H:1024 / 8 GB VRAM / 8 GB Shared

100%|█████████████████████████████████████████████████████████████████████████████████| 30/30 [00:41<00:00,  1.37s/it]
100%|█████████████████████████████████████████████████████████████████████████████████| 30/30 [06:54<00:00, 13.83s/it]
 10%|███████▊                                                                      | 1/10 [26:45<4:00:53, 1605.96s/it]

Is there anything that I maybe doing wrong. I have tried other controlnets but the processing time is not so much.

[EDIT] Can confirm it's depth that takes so much time. Without depth

100%|█████████████████████████████████████████████████████████████████████████████████| 30/30 [00:04<00:00,  6.06it/s]
100%|█████████████████████████████████████████████████████████████████████████████████| 10/10 [04:04<00:00, 24.48s/it]
Saving images: 100%|██████████████████████████████████████████████████████████████████| 30/30 [00:17<00:00,  1.74it/s]
Saving video: 100%|███████████████████████████████████████████████████████████████████| 30/30 [00:00<00:00, 51.22it/s]