search

kijai / ComfyUI-LivePortraitKJ

ComfyUI nodes for LivePortrait
MIT License
1.62k stars 123 forks source link

Animal mode #127

Open wapiters opened 2 months ago

wapiters commented 2 months ago

The animal mode doesn't seems to work.

I use the same image/video that I also used in pinokio livePortrait to make it speak and that works fine, but I can only animate the head while in your work flow i can use a full video.

error

kijai commented 2 months ago

MediaPipe is bad at detecting animal faces, try using the other croppers, insightface node also has detection threshold adjustment.

rslosch commented 2 months ago

None of the current croppers can detect well. Only works decently well with XPose.

Seryogerr commented 2 months ago

How to install X Pose? Please @kijai Yes i found N119 Comment , and guide how to install in on local liveportrait , but How to add this detector in ComfyUI? It's seems like i can't just git clone https://github.com/IDEA-Research/X-Pose.git in ComfyUI\custom_nodes\ComfyUI-LivePortraitKJ\face_alignment\detection git clone meme

kijai commented 2 months ago

To be completely honest, the reason it's not simple to install is why I have not wanted to support it. I have not attempted it yet myself either.

Seryogerr commented 2 months ago

To be completely honest, the reason it's not simple to install is why I have not wanted to support it. I have not attempted it yet myself either.

Sadly to hear that( Especially because X Pose works very good, i try it local by one-click-installer , and the quality of mouth rly good,especially the teeth and apparently it can be better for other unrealistic images like this https://github.com/KwaiVGI/LivePortrait/issues/108 and this https://github.com/KwaiVGI/LivePortrait/issues/111

With POWER of ComfyUI vid2vid it can be the best meme generator ever lol I don't want to rush you, but I just hope you find time to adapt it

Some examples with cat which were detected by sfd ~2/10 for sfd and 0/10 others

https://github.com/user-attachments/assets/06f5903b-4820-49db-8c5f-1ba88004523a

https://github.com/user-attachments/assets/bf1d219c-d36d-49cf-8235-9390b369730e

alenabkru commented 2 months ago

To be completely honest, the reason it's not simple to install is why I have not wanted to support it. I have not attempted it yet myself either.

thank you so much for your hard work. We will look forward to xpose. Because animating video to video with animals will be a big breakthrough.

davinci-jpg commented 2 months ago

X-pose installation is really too complicated, where can I find a tutorial?

Maxadox commented 1 month ago

How Can I use Animal mode in ComfyUI any tutorials ? Thanks

Icedcocon commented 1 month ago

I will continue to keep my eyes on this issue. Are there any plans to support LivePortrait animal models?

kijai commented 1 month ago

I will continue to keep my eyes on this issue. Are there any plans to support LivePortrait animal models?

Animal models have been supported since they came out, what's missing is the xpose detector that's better at detecting animal faces. It's possible to do some animal faces with just insightface if you turn down the detection threshold.

I don't currently have time to implement xpose, and I have no use for it myself. Biggest issue with it is that it's not the easiest thing to install, supporting it in the nodes probably wouldn't be too hard as long as the user has it installed.