search

aigc-apps / sd-webui-EasyPhoto

📷 EasyPhoto | Your Smart AI Photo Generator.
Apache License 2.0
4.98k stars 399 forks source link

Add LCM, IPA, and Scene Lora in Video Inference. #324

Closed bubbliiiing closed 11 months ago

bubbliiiing commented 11 months ago

  1. LCM New LCM acceleration added. When producing images of 512x512 resolution, totaling 16 frames, with all models preheated, the generation time is approximately 100 seconds. image

  2. IPA Added IPA control that can improve likeness. After usage, the similarity of portraits is mostly above 0.65. image

  3. Scene Lora Added a new scene, Lora, which is used in the same way as text2photo. image 00000039

wuziheng commented 11 months ago

commit aff26f0a0 has bug 截屏2023-12-11 下午7 26 19

should fix @bubbliiiing

bubbliiiing commented 11 months ago

commit aff26f0 has bug 截屏2023-12-11 下午7 26 19

should fix @bubbliiiing

已经修复。 这是因为在scripts/easyphoto_utils/animatediff/animatediff_infv2v.py的125行，获取了control.control_model.uncond_image_emb[context]，但uncond_image_emb事实上长度只有1，并没有拓展到可以获取的长度； 而与之对应的control.control_model.image_emb的长度是生成视频帧数的2倍，使用context可以正常获取。

如图为controlnet中的定义，具体为何为1还需debug，因为看起来在controlnet定义中，正负两个的长度应该一样： image

wuziheng commented 11 months ago

768*512 16frame + superresolution on A10 W/O LCM ： 250s W LCM ：155s