openvinotoolkit / stable-diffusion-webui

Stable Diffusion web UI
GNU Affero General Public License v3.0
250 stars 39 forks source link

[Feature Request]: Add support for keyword weights #43

Closed sjrummins closed 10 months ago

sjrummins commented 10 months ago

Is there an existing issue for this?

What would your feature do ?

Thanks very much for adding OpenVINO support, I really appreciate it - I've been playing around with Stable Diffusion on the command line using OpenVINO (as I don't have a graphics card) but it's so much nicer to have the web GUI.

It seems to me that keyword weights are not working with OpenVINO acceleration. For example, I've attached two images generated using the same prompt and seed:

Street scene with (dog:1.5)

The idea here is to give extra weight to the "dog" keyword in the prompt.

The 00205-42-openvino one is generated with the "Accelerate with OpenVINO" script, the 00206-42-cpu one is generated without it (running on my CPU). They are quite different, whereas if I use a prompt without this kind of prompt weighting the results are nearly (but not quite) identical with or without OpenVINO acceleration as I'd expect - just much quicker with the acceleration.

I've tried to offer this as a simple example. I find playing around with more complex prompts using keyword weighting makes the fact that this isn't working more obvious, but it's not so easy to show in an issue.

I've tried this with commit 60b70401343d09cb4ae58d720c253eefff423ea7, which is the latest as I write this.

It would be really nice if this could be supported in the standard way.

00205-42-openvino 00206-42-cpu

Proposed workflow

  1. Use standard "(foo:1.5)", "((foo))", "[foo]" syntax in prompts
  2. The terms marked in this way are emphasised or de-emphasised when the prompt is processed.

Additional information

No response

qiacheng commented 10 months ago

Hi @sjrummins, thanks for the suggestion. prompt weighting is now added and PR is created : https://github.com/openvinotoolkit/stable-diffusion-webui/pull/46

SD v1.5

Seed: 4168475539

prompt: a [[red cat]] playing with a ball image

prompt: a [[red cat]] playing with a (ball:1.5) image