Avoiding Model Interference When Using set_options(/sdapi/v1/options) in a Multi-User Environment

[zhixideyu]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What happened?

Hello,

I would like to address an issue related to the usage of set_options(/sdapi/v1/options) in a multi-user environment. As my API is being used by multiple individuals simultaneously, I have noticed a specific scenario that requires attention.

Let me provide an example:

User A opens the webpage and selects the SD1.5 model to load an image. Meanwhile, User B utilizes the API interface to set the options and chooses the SD2.0 model.

In this situation, even though User A sees that they have selected the SD1.5 model on the webpage, the actual inference is performed using the SD2.0 model due to User B's subsequent selection.

My question is: How can we ensure that using set_options(/sdapi/v1/options) does not impact the model already chosen on the webpage?

Thank you for your assistance.

Best regards,

Steps to reproduce the problem

1. User A opens the webpage and selects the SD1.5 model to load an image. Meanwhile,
2. User B utilizes the API interface to set the options and chooses the SD2.0 model.

What should have happened?

User A still uses sd1.5 correctly

Commit where the problem happens

NO

What Python version are you running on ?

Python 3.10.x

What platforms do you use to access the UI ?

Windows

What device are you running WebUI on?

Other GPUs

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

NO

List of extensions

NO

Console logs

NO

Additional information

NO

[w-e-w]

I don't think we design this with simultaneous multi-user in mind

[huchenlei]

I think one solution could be attaching the model info to the StableDiffusionProcessing. Switching model if the loaded one is not the one specified in the StableDiffusionProcessing.

By doing so the model selection on WebUI can be lazy, i.e. not loading checkpoint each time the selection value changes.

[w-e-w]

if the issue is just models loading that I think the best solution would be to just set the model override as opposed to select model

override information can select a model to be used which will be loaded when that particular request is processed webui then switch the model to the correct model

I think in the way this is what huchenlei is suggesting

the model will only get Switched At the beginning of the generation

[walmt]

if the issue is just models loading如果问题只是模型加载 that I think the best solution would be to just set the model override我认为最好的解决方案是只设置模型覆盖 as opposed to select model与选择模型相反

override information can select a model to be used which will be loaded when that particular request is processed覆盖信息可以选择要使用的模型，在处理特定请求时加载 webui then switch the model to the correct model将webui切换到正确的模型

I think in the way this is what huchenlei is suggesting我认为胡晨雷是在暗示这个意思

the model will only get Switched At the beginning of the generation该模型只会在生成的开始时进行切换

Hello! I would like to know when you will be adding this model override option in the project? I tried to add it myself in the code, but as you understand, the codebase is quite large, and it's indeed time-consuming to read for someone who is not very familiar with Python. However, this feature is urgently needed. I noticed that many people have mentioned this in the Issue, and everyone seems to really need this feature.

[w-e-w]

Hello! I would like to know when you will be adding this model override option in the project? I tried to add it myself in the code, but as you understand, the codebase is quite large, and it's indeed time-consuming to read for someone who is not very familiar with Python. However, this feature is urgently needed. I noticed that many people have mentioned this in the Issue, and everyone seems to really need this feature.

override settings (which includes overriding model) has been a feature for a very long time before this post is even made

[w-e-w]

quick example base on api example of o the wiki

import requests
import io
import base64
from PIL import Image, PngImagePlugin
import time

url = "http://localhost:55555"

payload = {
    "prompt": "1girl",
    "seed": 1,
    "steps": 20,
    "override_settings": {"sd_model_checkpoint": "5d4e1dcb95"},  #  <<<----------------- "sd_model_checkpoint": ""
}

response = requests.post(url=f'{url}/sdapi/v1/txt2img', json=payload)

r = response.json()

for i in r['images']:
    image = Image.open(io.BytesIO(base64.b64decode(i.split(",",1)[0])))

    png_payload = {
        "image": "data:image/png;base64," + i
    }
    response2 = requests.post(url=f'{url}/sdapi/v1/png-info', json=png_payload)

    pnginfo = PngImagePlugin.PngInfo()
    pnginfo.add_text("parameters", response2.json().get("info"))
    image.save(f'output{time.time()}.png', pnginfo=pnginfo)

you don't have to use hash, midel filename also works "sd_model_checkpoint": "sd_xl_base_1.0"

it is possible to override it to most settings, not just models

[walmt]

quick example base on api example of o the wiki

import requests
import io
import base64
from PIL import Image, PngImagePlugin
import time

url = "http://localhost:55555"

payload = {
    "prompt": "1girl",
    "seed": 1,
    "steps": 20,
    "override_settings": {"sd_model_checkpoint": "5d4e1dcb95"},  #  <<<----------------- "sd_model_checkpoint": ""
}

response = requests.post(url=f'{url}/sdapi/v1/txt2img', json=payload)

r = response.json()

for i in r['images']:
    image = Image.open(io.BytesIO(base64.b64decode(i.split(",",1)[0])))

    png_payload = {
        "image": "data:image/png;base64," + i
    }
    response2 = requests.post(url=f'{url}/sdapi/v1/png-info', json=png_payload)

    pnginfo = PngImagePlugin.PngInfo()
    pnginfo.add_text("parameters", response2.json().get("info"))
    image.save(f'output{time.time()}.png', pnginfo=pnginfo)

you don't have to use hash, midel filename also works "sd_model_checkpoint": "sd_xl_base_1.0"

it is possible to override it to most settings, not just models

Is this the API calling pattern? My goal is to deploy StableDiffusionWebUI on my GPU server, and then allow others to access it directly via my IP:port. Afterwards, when they generate content, it will be based on the model they choose, and they will not be affected by others switching models. How should I modify your code to achieve this?

[w-e-w]

https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/API

[van1027]

Yep, I experienced the same behaviour, just could not point a finger what's causing it. Even worse, the txt file that contains prompt tells that Model A has been used for generation, when in fact it used Model B, selected by the other user. So you cannot reproduce your image as you don't know which model has been used for generation.

[walmt]

overriding

I still believe that developing through API interfaces can be a cumbersome task, at least when it comes to addressing this particular issue. Developing through APIs often results in losing some functionalities available on the web UI since APIs cannot perfectly replicate all the content of the web UI.

In my opinion, the logic to support model switching for multiple users should be modified within the web UI.

For instance, continuously check which model is selected in the current web UI. If it differs from the global model, users can switch to it. This switching process can be synchronized with the image generation time, allowing multiple users to effectively utilize the web UI in scenarios with relatively low concurrency.

[zmhgit]

选择模型

覆盖信息

May I ask how to solve this problem? I have encountered the same problem, multi-user, and multi-user is A waiting process, A requests first, and then b, but the execution is sequential, B needs to wait for A to return the image before execution

[w-e-w]

payload: {
    "override_settings": {
         'sd_model_checkpoint': "sd_xl_base_1.0",  # this can use to switch sd model
    },
}