basujindal / stable-diffusion

Optimized Stable Diffusion modified to run on lower GPU VRAM
Other
3.14k stars 469 forks source link

Weighting prompts: how is it supposed to work? #177

Open salamanders opened 1 year ago

salamanders commented 1 year ago

I was trying to build up a cartoon for a school of a black panther wearing a backpack

  1. Dall-E does a great job: "Cartoon of a panther wearing a school backpack. The panther is walking into a middle school excited for the first day of class" firstday
  2. and I was attempting the same in SD. black panther wearing a backpack was very Disney, Wakanda Forever! Chadwick Boseman wearing a backpack.
  3. So I started weighting the terms, and black panther:1.0 wearing:1.0 a backpack:0.9 is a mess! Random people with t-shirts with a black panther on it.

Still nuts. black panther:0.9 wearing:0.9 a backpack:0.9 white background clipart download

xcorvis commented 1 year ago

I was told in another thread the weight acts as a separator, so your prompt (minus weights) becomes "black panther, wearing, a backpack, white background clipart". I believe 1.0 is also the default weight, so there's no need to specify that.

"Black panther" is obviously triggering on the movie, so you could try just "panther" (many people who use the term panther assume it means black anyway), or specify black elsewhere, or "black furred panther" and so on. Maybe even "a black panther" would reduce the hits for movie Black Panther.

mahlonsmith commented 1 year ago

Out of curiosity, are negative weights also supported?

salamanders commented 1 year ago

And a very basic question: is "no weight" the same as "some words:0.5"?

xcorvis commented 1 year ago

I'm definitely not an expert, but I believe negative weights are not supported. According to the code comments, "no weight" is equivalent to 1.0.