search

KwaiVGI / LivePortrait

Bring portraits to life!
https://liveportrait.github.io
Other
12.9k stars 1.37k forks source link

Some expressions don't work. Is it possible to fine tune? #166

Open HumanGamer opened 3 months ago

HumanGamer commented 3 months ago

I noticed that if the driving video makes expressions where they stick their tongue out, such as making silly faces, it doesn't work.

There may be other expressions that don't work, but this is what I noticed so far.

Can this be fixed in a future update? Or is it possible to fine tune the model to make it support this?

Mrkomiljon commented 3 months ago

@HumanGamer How about your driver video resolution?

HumanGamer commented 3 months ago

@HumanGamer How about your driver video resolution?

512x512

https://github.com/user-attachments/assets/59324bf9-82bd-4131-b869-a1fc5c1285dd

Here is an example of the result (would prefer not to share the driving video with my face)

When I stick out my tongue, the result just moves the mouth kinda weird.

AlonDan commented 3 months ago

I can only guess, but maybe the original model trained without tongue out?

HumanGamer commented 3 months ago

I can only guess, but maybe the original model trained without tongue out?

That's what I was thinking. I wonder if we can fine tune the model?

galigaligo commented 3 months ago

It is basically impossible to achieve. Firstly, there is a high probability that there will be no tongue sticking out in his training data. Secondly, there is currently no key point acquisition method that can capture tongue key points