[Feature Request] FaceID-Lora mask

[MoonMoon82]

I always had the issue, that after composing the prompt for an image, as soon I add the FaceID Lora, the resulting image looks very different than befor.

For example, if I want to prompt for "beautiful female model posing, wearing a wedding dress" without the FaceID Lora/Adapter it is resulting in an image like this:

(image 1)

Now if want to add this face without changing the seed

(image 2)

the result after attaching the FaceID Lora + FaceID IPAdapter looks like this:

(image 3)

Now I just found out, that it is possible to mask the target area of a lora by using AnimateDiffEvolved nodes ADE_RegisterLoraHook + ADE_ConditioningSetMaskAndCombinenode

With these nodes it is possible to restrict the influence of a lora - in this case the masked face area.

(image 4)

As you can see, image 4 is much more like image 1 than image 3. Of course in general you can also lower the strength of the FaceID lora model, but this effects the influence of the FaceID-IPAdapter.

To show you, that it already works, I attach the following SDXL workflow: LoraMask.json Only dependencies in my workflow as far as I can see:

• AnimateDiffEvolve
• WAS Node Suit
• IPAdapter_Plus

It would be great if you could consider to add such mask that also restricts the influence of the FaceID-Lora.

Kind regards!

[cubiq]

have you tried with attention mask?

[MoonMoon82]

@cubiq Yes, in my example workflow, this is why I placed the mask reroute near to the IPAdapter:

With connected attention mask the output looks also very different to the image without FaceID-Lora/IPAdapter:

To restrict the influence of the FaceID-Lora I really recommend to use the AnimateDiffEvolve nodes like in the example workflow! I created this feature request to show you that this would be an awesome feature to be implemented into your IPAdapter nodes.

[cubiq]

only one note, the lora weight should be around .6. 1 is too strong.

apart from that there seems already a way to mask the lora, are you asking to integrate it into the unified loader?

[MoonMoon82]

@cubiq Yes, the value in my example is too high, I just wanted to show the influences of the FaceID Lora.

If it is possible to implement it as an optional mask input, I guess the IPAdapterUnifiedLoaderFaceID would be the right place: In case you want to have multiple masked IPAdapterFaceID in your workflow, then you can just connect a merged mask to the IPAdapterUnifiedLoaderFaceID. I guess this would be a great solution!