Suggestion / question: Single image as "drivers"?

[DL420]

Would it be possible to have a feature where you simply use one still image of an expression affecting the target image and creating a tweaked output?

Within the field of image manipulation there are many techniques that people use to "tease out" expressions. From simply text prompting for them ("picture of a happy person") or using an array of other methods such as controlnets, ipadapter, various img2img work flows and so on.

Since Liveportrait can use video to rearrange "key points" on a face without doing a face swap, might it be possible to use a single image as a driver? That would mean a lot for expression genning I think, since you could just pick an expression that you like, from whatever source, and then experiment with that applied to another image without having to go through the various processes mentioned above.

[aihacker111]

Yes you can , because through the process of live-portrait , the model central of this is spade generator and warping mosel , both of it is a desgin with GAN network, so if follow your idea that mean use the affecting image to change the another image follow it can be possible, but i think we don’t need to train the new model for this, just follow the start key and end key frame (we can modify it from live portrait) and can control it by prompt like zero-shot learning, i think it’ll work

[jetwaves]

+1

driver videos is cut to frames before extraction of emotion, a simple code branch can do that .

Would it be possible to have a feature where you simply use one still image of an expression affecting the target image and creating a tweaked output?

Within the field of image manipulation there are many techniques that people use to "tease out" expressions. From simply text prompting for them ("picture of a happy person") or using an array of other methods such as controlnets, ipadapter, various img2img work flows and so on.

Since Liveportrait can use video to rearrange "key points" on a face without doing a face swap, might it be possible to use a single image as a driver? That would mean a lot for expression genning I think, since you could just pick an expression that you like, from whatever source, and then experiment with that applied to another image without having to go through the various processes mentioned above.

[DL420]

I would be nice. A simple, intuitive way to try out various expressions on generated images that you really like. You have your epic "cyber dude" generated, and now you can slap various expressions on him without having to go into one of the numerous more complicated and time consuming techniques that people use, in Stable Diffusion and so on.

[aihacker111]

@DL420 Stable Diffusion really nice but it’s not a choice for everyone, at least for us, now diffusion is usually use by some big tech and we don’t have any super computer to run it, we know what it does, how it run, about the design diffusion (unet, vae, sheduler,…) , but we don’t have computer to run it, we also just run it with cpu off load or fp16 and finally it just stop for fun Users need the faster model can run in the feel time and it have low cost to deploy this model, so with your idea , I’m very intersting

[x4080]

Yeah, this is very useful idea

[singlinhaha]

Have you implemented this feature yet？

[aihacker111]

@singlinhaha I'm also to do that but have some issues with connect prompt, but the idea can be completed

[singlinhaha]

@aihacker111 Thank you for your answer. Looking forward to your work!!! I am a beginner in this field.

[DL420]

@singlinhaha I'm also to do that but have some issues with connect prompt, but the idea can be completed

Glad to hear you've found a way to implement this! Loving you your work!