search

lllyasviel / Fooocus

Focus on prompting and generating
GNU General Public License v3.0
38.1k stars 5.09k forks source link

Faceswap and loras #1997

Open techalexx opened 5 months ago

techalexx commented 5 months ago
  1. faceswap should be the last stage and not the first, otherwise the connected loras will distort the portrait resemblance. Actually this is a bug!
  2. need a combined weight for image prompt for all connected photos at the same time
  3. store image prompt presets for next jobs
mashb1t commented 5 months ago

Hey @techalexx,

  1. can't verify if this is intended or not, maybe @lllyasviel is able to help here
  2. feel free to provide a PR for this
  3. please elaborate, the prompt/styles aren't deleted after clicking generated (between jobs). Do you refer to closing and opening the tab of Fooocus? If yes, this issue (enhancement) already exists, see https://github.com/lllyasviel/Fooocus/issues/1622, but is in contrast with the concept of presets.
techalexx commented 5 months ago

  1. this is important if you don't want to change the balance between several photos but need to reduce the overall weight. 03

  2. that's not exactly what I meant. since gradio can not to save presets, this can be done using config.txt : Expand settings. Add settings for images, for random and seed and etc. Also add the ability to set multiple paths in one key.

  3. new feature 04

raymondgp commented 5 months ago
  1. faceswap should be the last stage and not the first, otherwise the connected loras will distort the portrait resemblance. Actually this is a bug!
  2. need a combined weight for image prompt for all connected photos at the same time
  3. store image prompt presets for next jobs

Consider if you run the faceswap last then you cannot apply styles. The faceswap will usually keep whatever original style comes from the source. If you are trying to swap a face from a photo into say, a cartoon, the face will look mostly as a photo merged into a cartoon.

skylerblack2 commented 5 months ago
  1. faceswap should be the last stage and not the first, otherwise the connected loras will distort the portrait resemblance. Actually this is a bug!
  2. need a combined weight for image prompt for all connected photos at the same time
  3. store image prompt presets for next jobs

Consider if you run the faceswap last then you cannot apply styles. The faceswap will usually keep whatever original style comes from the source. If you are trying to swap a face from a photo into say, a cartoon, the face will look mostly as a photo merged into a cartoon.

I think as a general principle you want face swap to prioritize fidelity over blending with the style of the surrounding image. If the likeness is high fidelity, blending it with the surrounding image in subsequent passes is much easier than dealing with a lower fidelity image due to whatever happens after swapping. You can always do another pass on the swapped image and apply a style afterwards.

But if the output image doesn't look like the individual being swapped, then the image is unusable no matter how beautifully blended it is.

That said, I'm not sure that the implementation of face swap fooocus uses works this way, where face swap is a distinct step. I get the sense that the likeness is being applied at every step throughout the generation. So I'm not sure this is a bug either.

techalexx commented 5 months ago

Even if this is not a bug, it is a flaw. This cannot be used

barepixels commented 4 months ago

I think InstantID solve this problem with Lora not sure if photobooth too