The issue of facial contamination in LoRA

[chaorenai]

In the character Lora, if the output is a group photo, the face of the character Lora contaminates the faces of other people in the group. Various methods such as adjusting the dataset, lowering the learning rate, and layer-wise training have been tried, but the issue cannot be resolved. What exactly is going wrong?

[Orenji-Tangerine]

Generate a group photo without LoRA and change the face of that particular person by using inpainting in image to image with LoRA.

[chaorenai]

Generate a group photo without LoRA and change the face of that particular person by using inpainting in image to image with LoRA. The functionality can be achieved through secondary face-swapping using InstantID or flux-Lora. However, it is hoped that the trained Lora model can directly produce group photos, which could solve the issue of facial distortion.

[Orenji-Tangerine]

LoRA changes the idea of the base models and defines how every person looks like. lnpainting is the only way.