search

aihacker111 / Efficient-Live-Portrait

Fast running Live Portrait with TensorRT and ONNX models
MIT License
122 stars 10 forks source link

Request Feature: Image to image driving #20

Open Mayorc1978 opened 1 month ago

Mayorc1978 commented 1 month ago

I asked for this in the original repo:

FEATURE REQUEST - Possibility to pose a single image using another image (not a video) as ref and then retargeting using controls for refinement for eyes/mouth/pitch...

As a general implementation not necessarily tied to Comfy/SD.

Obviously the first part is the most important and that is driving an image using another image (not a video), the second part is to make a pose (face pose and expression) template maker (currently original repo does motion template, since it uses video for driving not images). This second part (the template maker) would allow to create a db of face expressions, similar to the way 3d animation software create those poses for phonemes and expressions and then keyframe animate or interpolate from those. It would also be useful for activities like batch image posing (since the pose-espression is already in a template) in a efficient way.

aihacker111 commented 1 month ago

@Mayorc1978 I’m also finished this code, but many day ago , i’m too busy, I’ll public img2img code tonight , stay tuned!

Mayorc1978 commented 1 month ago

@Mayorc1978 I’m also finished this code, but many day ago , i’m too busy, I’ll public img2img code tonight , stay tuned!

That's great! Does it work on the fly or like someone did they drove the the frames iteratively to be sure to match the emotion/expression as close as possible?

Mayorc1978 commented 1 month ago

@Mayorc1978 I’m also finished this code, but many day ago , i’m too busy, I’ll public img2img code tonight , stay tuned!

Any update?