[Bug]: '422 Client Error: Unprocessable Entity for url POST img2img

alexsluborski77 commented 4 months ago

Checklist

[ ] The issue exists after disabling all extensions
[ ] The issue exists on a clean installation of webui
[ ] The issue is caused by an extension, but I believe it is caused by a bug in the webui
[ ] The issue exists in the current version of the webui
[ ] The issue has not been reported before recently
[ ] The issue has been reported before but has not been fixed yet

What happened?

I am running auto1111 in docker container and hitting it through another web server exposed through application load balancer. I have the auto1111 enabled with api and I can confirm everything works fine and is loaded but I am having trouble getting my payload request through for img2img render endpoints.

This is the curl method im using with Windows Powershell currently as that is the machine I have access to: Invoke-RestMethod -Uri 'https://load-balancer-url/render5' -Method Post -Headers @{ 'Access-Token' = "$accessToken"; 'From'= 'user@email'; 'accept' = 'application/json'; 'Content-Type' = 'application/json' } -Body $jsonData -Verbose

http://127.0.0.1:7860/sdapi/v1/img2img is what the server redirects to since it is in docker container network but that load-balancer is used for accessing from internet.

When I run this command I get this error: Render failed due to the following error: Error: ('422 Client Error: Unprocessable Entity for url') The access token and email is correct and working fine (JWT token)

This is my json payload...

PS D:\Repositories> Write-Host $jsonData
{
    "atmospheric":  false,
    "mask":  null,
    "edge_map":  null,
    "prompt":  "turkey sub",
    "prompt_strength":  60,
    "geometry_override":  75,
    "height":  768,
    "number_of_renderings":  1,
    "aerial":  false,
    "width":  768,
    "depth_map":  null,
    "seed":  -1,
    "image":  "data:image/png;base64,iVBORw...",
    "material_override":  100
}

I can't get any more verbose error logging either from the --api-log or from container logs as well as ssh as to whats wrong with the response body. Could anyone assist as to what its complaining about? What is the unprocessable entity here? I am using release-candidate branch here with commit hash cf2772fab0af5573da775e7437e6acdca424f26e.

This is the docker file with command line args

FROM alpine/git:2.36.2 as download

COPY clone.sh /clone.sh

RUN . /clone.sh stable-diffusion-stability-ai https://github.com/Stability-AI/stablediffusion.git cf1d67a6fd5ea1aa600c4df58e5b47da45f6bdbf \
  && rm -rf assets data/**/*.png data/**/*.jpg data/**/*.gif

RUN . /clone.sh CodeFormer https://github.com/sczhou/CodeFormer.git c5b4593074ba6214284d6acd5f1719b6c5d739af \
  && rm -rf assets inputs

RUN . /clone.sh BLIP https://github.com/salesforce/BLIP.git 48211a1594f1321b00f14c9f7a5b4813144b2fb9
RUN . /clone.sh k-diffusion https://github.com/crowsonkb/k-diffusion.git ab527a9a6d347f364e3d185ba6d714e22d80cb3c
RUN . /clone.sh clip-interrogator https://github.com/pharmapsychotic/clip-interrogator 2cf03aaf6e704197fd0dae7c7f96aa59cf1b11c9
RUN . /clone.sh generative-models https://github.com/Stability-AI/generative-models 45c443b316737a4ab6e40413d7794a7f5657c19f

FROM pytorch/pytorch:2.1.2-cuda12.1-cudnn8-runtime

ENV DEBIAN_FRONTEND=noninteractive PIP_PREFER_BINARY=1

RUN --mount=type=cache,target=/var/cache/apt \
  apt-get update && \
  # we need those
  apt-get install -y fonts-dejavu-core rsync git jq moreutils aria2 \
  # extensions needs those
  ffmpeg libglfw3-dev libgles2-mesa-dev pkg-config libcairo2 libcairo2-dev build-essential

WORKDIR /
RUN --mount=type=cache,target=/root/.cache/pip \
  git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git && \
  cd stable-diffusion-webui && \
  git reset --hard cf2772fab0af5573da775e7437e6acdca424f26e && \
  pip install -r requirements_versions.txt

ENV ROOT=/stable-diffusion-webui

COPY --from=download /repositories/ ${ROOT}/repositories/
RUN mkdir ${ROOT}/interrogate && cp ${ROOT}/repositories/clip-interrogator/clip_interrogator/data/* ${ROOT}/interrogate
RUN --mount=type=cache,target=/root/.cache/pip \
  pip install -r ${ROOT}/repositories/CodeFormer/requirements.txt

RUN --mount=type=cache,target=/root/.cache/pip \
  pip install pyngrok xformers==0.0.23.post1 \
  git+https://github.com/TencentARC/GFPGAN.git@8d2447a2d918f8eba5a4a01463fd48e45126a379 \
  git+https://github.com/openai/CLIP.git@d50d76daa670286dd6cacf3bcd80b5e4823fc8e1 \
  git+https://github.com/mlfoundations/open_clip.git@v2.20.0

# there seems to be a memory leak (or maybe just memory not being freed fast enough) that is fixed by this version of malloc
# maybe move this up to the dependencies list.
RUN apt-get -y install libgoogle-perftools-dev && apt-get clean
ENV LD_PRELOAD=libtcmalloc.so

COPY . /docker

RUN \
  # mv ${ROOT}/style.css ${ROOT}/user.css && \
  # one of the ugliest hacks I ever wrote \
  sed -i 's/in_app_dir = .*/in_app_dir = True/g' /opt/conda/lib/python3.10/site-packages/gradio/routes.py && \
  git config --global --add safe.directory '*'

WORKDIR ${ROOT}
ENV NVIDIA_VISIBLE_DEVICES=all
EXPOSE 7860
RUN chmod +x /docker/entrypoint.sh
ENTRYPOINT ["/docker/entrypoint.sh"]
CMD python -u webui.py --api --listen --port 7860 --allow-code --xformers --enable-insecure-extension-access --nowebui --no-half-vae --api-log

Steps to reproduce the problem

I run docker container with this file and attempt the curl command and I get this error with no other verbose logging. with --verbose flag on curl I get something like this but not much more helpful.

CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-RestMethod], WebException
- FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeRestMethodCommand

What should have happened?

It should process POST request with JSON payload succesfully.

What browsers do you use to access the UI ?

Google Chrome

Sysinfo

Not using WebUI

Console logs

This is from the container logs...

total 648K                                                                                                                                          
drwxr-xr-x.  1 root root   25 Jun  4 10:02 .                                                                                                        
drwxr-xr-x.  1 root root   51 Jun  4 10:30 ..                                                                                                       
-rw-r--r--.  1 root root   48 Jun  4 10:02 .eslintignore                                                                                            
-rw-r--r--.  1 root root 3.4K Jun  4 10:02 .eslintrc.js
drwxr-xr-x.  8 root root  163 Jun  4 10:02 .git
-rw-r--r--.  1 root root   55 Jun  4 10:02 .git-blame-ignore-revs
drwxr-xr-x.  4 root root   77 Jun  4 10:02 .github
-rw-r--r--.  1 root root  495 Jun  4 10:02 .gitignore
-rw-r--r--.  1 root root  119 Jun  4 10:02 .pylintrc
-rw-r--r--.  1 root root  51K Jun  4 10:02 CHANGELOG.md
-rw-r--r--.  1 root root  243 Jun  4 10:02 CITATION.cff
-rw-r--r--.  1 root root  657 Jun  4 10:02 CODEOWNERS
-rw-r--r--.  1 root root  35K Jun  4 10:02 LICENSE.txt
-rw-r--r--.  1 root root  12K Jun  4 10:02 README.md
drwxr-xr-x.  2 root root  172 Jun  4 10:02 configs
drwxr-xr-x.  2 root root   57 Jun  4 10:02 embeddings
-rw-r--r--.  1 root root  167 Jun  4 10:02 environment-wsl2.yaml
drwxr-xr-x.  2 root root   37 Jun  4 10:02 extensions
drwxr-xr-x. 11 root root  175 Jun  4 10:02 extensions-builtin
drwxr-xr-x.  2 root root  141 Jun  4 10:02 html
drwxr-xr-x.  2 root root  104 Jun  4 10:02 interrogate
drwxr-xr-x.  2 root root 4.0K Jun  4 10:02 javascript
-rw-r--r--.  1 root root 1.4K Jun  4 10:02 launch.py
drwxr-xr-x.  2 root root   45 Jun  4 10:02 localizations
drwxr-xr-x.  7 root root   89 Jun  4 10:02 models
drwxr-xr-x.  8 root root 4.0K Jun  4 10:02 modules
-rw-r--r--.  1 root root  185 Jun  4 10:02 package.json
-rw-r--r--.  1 root root  813 Jun  4 10:02 pyproject.toml
drwxr-xr-x.  8 root root  142 Jun  4 08:26 repositories
-rw-r--r--.  1 root root   49 Jun  4 10:02 requirements-test.txt
-rw-r--r--.  1 root root  360 Jun  4 10:02 requirements.txt
-rw-r--r--.  1 root root  556 Jun  4 10:02 requirements_versions.txt
-rw-r--r--.  1 root root 411K Jun  4 10:02 screenshot.png
-rw-r--r--.  1 root root 5.3K Jun  4 10:02 script.js
drwxr-xr-x.  2 root root 4.0K Jun  4 10:02 scripts
-rw-r--r--.  1 root root  23K Jun  4 10:02 style.css
drwxr-xr-x.  3 root root  151 Jun  4 10:02 test
drwxr-xr-x.  2 root root  138 Jun  4 10:02 textual_inversion_templates
-rw-r--r--.  1 root root  670 Jun  4 10:02 webui-macos-env.sh
-rw-r--r--.  1 root root   84 Jun  4 10:02 webui-user.bat
-rw-r--r--.  1 root root 1.4K Jun  4 10:02 webui-user.sh
-rw-r--r--.  1 root root 2.3K Jun  4 10:02 webui.bat
-rw-r--r--.  1 root root 5.4K Jun  4 10:02 webui.py
-rwxr-xr-x.  1 root root 7.9K Jun  4 10:02 webui.sh
./
model.pt
./
ViT-L-14_stats.th
Mounted embeddings
Mounted .cache
Mounted styles.csv
Mounted ui-config.json
Mounted models
Mounted .cache
Mounted config_states
Mounted config.json
Mounted extensions
Installing extension dependencies (if any)
Calculating sha256 for /stable-diffusion-webui/models/Stable-diffusion/512-depth-ema.safetensors: Startup time: 9.5s (import torch: 3.8s, import gradio: 1.2s, setup paths: 1.8s, initialize shared: 0.5s, other imports: 0.8s, setup codeformer: 0.1s, list SD models: 0.2s, load scripts: 0.7s, lora_script.py: 0.2s).
INFO:     Started server process [72]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:7860 (Press CTRL+C to quit)
59cb052d4409654936139693b57e4632d119bdd38df9a3cdf30d4a63799a4853
Loading weights [59cb052d44] from /stable-diffusion-webui/models/Stable-diffusion/512-depth-ema.safetensors
Creating model from config: /stable-diffusion-webui/repositories/stable-diffusion-stability-ai/configs/stable-diffusion/v2-midas-inference.yaml     
Applying attention optimization: xformers... done.
Model loaded in 29.1s (calculate hash: 23.1s, load weights from disk: 0.4s, create model: 3.1s, apply weights to model: 1.7s, load textual inversion embeddings: 0.4s, calculate empty prompt: 0.1s).
API 2024-06-04 18:44:26.011080 422 http/1.1 POST /sdapi/v1/img2img 127.0.0.1 0.0286
INFO:     127.0.0.1:60596 - "POST /sdapi/v1/img2img HTTP/1.1" 422 Unprocessable Entity

w-e-w commented 4 months ago

your payload does not look anything like a web you are img2img payload it should look something like this

{
  "prompt": "",
  "negative_prompt": "",
  "styles": [
    "string"
  ],
  "seed": -1,
  "subseed": -1,
  "subseed_strength": 0,
  "seed_resize_from_h": -1,
  "seed_resize_from_w": -1,
  "sampler_name": "string",
  "scheduler": "string",
  "batch_size": 1,
  "n_iter": 1,
  "steps": 50,
  "cfg_scale": 7,
  "width": 512,
  "height": 512,
  "restore_faces": true,
  "tiling": true,
  "do_not_save_samples": false,
  "do_not_save_grid": false,
  "eta": 0,
  "denoising_strength": 0.75,
  "s_min_uncond": 0,
  "s_churn": 0,
  "s_tmax": 0,
  "s_tmin": 0,
  "s_noise": 0,
  "override_settings": {},
  "override_settings_restore_afterwards": true,
  "refiner_checkpoint": "string",
  "refiner_switch_at": 0,
  "disable_extra_networks": false,
  "firstpass_image": "string",
  "comments": {},
  "init_images": [
    "string"
  ],
  "resize_mode": 0,
  "image_cfg_scale": 0,
  "mask": "string",
  "mask_blur_x": 4,
  "mask_blur_y": 4,
  "mask_blur": 0,
  "mask_round": true,
  "inpainting_fill": 0,
  "inpaint_full_res": true,
  "inpaint_full_res_padding": 0,
  "inpainting_mask_invert": 0,
  "initial_noise_multiplier": 0,
  "latent_mask": "string",
  "force_task_id": "string",
  "sampler_index": "Euler",
  "include_init_images": false,
  "script_name": "string",
  "script_args": [],
  "send_images": true,
  "save_images": false,
  "alwayson_scripts": {},
  "infotext": "string"
}

if you have trouble creating payload yourself then use the Web Interface + https://github.com/huchenlei/sd-webui-api-payload-display to to generate the payload for you

422 is most likely trigger by your specifying a non-existent script / extension

alexsluborski77 commented 4 months ago

Ok thanks, I was missing a big chunk of the payload, I think this a problem due to controlnet script

 "alwayson_scripts": {
            "controlnet": {
                "args": controlNetArgs

located at the end of the payload. I have to look in docs to confirm how it can find this script, im sure it is outlined somewhere? If by non-existent, you mean it is not in the file system, there is a specific folder to place?

alexsluborski77 commented 4 months ago

I figured it out, was not installing in the right directory needs to be in data folder in extensions->controlnet->models. I was getting the controlnet from hugging face. Is it possible to try add more verbose logging about issues with payload or missing scripts in api.py related to 422?

w-e-w commented 4 months ago

Is it possible to try add more verbose logging about issues with payload or missing scripts in api.py related to 422?

thing is we did put "more verbose logs", I'm not sure why but the the more useful messages is not displayed by the client.

there are three 422 inside modules/api/api.py this is 2/3 https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/feee37d75f1b168768014e4634dcb156ee649c05/modules/api/api.py#L351-L355

w-e-w commented 4 months ago

I have to look in docs to confirm how it can find this script, im sure it is outlined somewhere? If by non-existent, you mean it is not in the file system, there is a specific folder to place?

I'm afraid we don't have good documentations

use https://github.com/huchenlei/sd-webui-api-payload-display It's relatively easy to learn with actual examples of a payload

the best documentation we have in house is this https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/API I have example script that you can try, I recall I also included a control-net payload example but it's most likely outdated so you most likely will still have to create your new payload using sd-webui-api-payload-display

AUTOMATIC1111 / stable-diffusion-webui