search

huchenlei / sd-webui-openpose-editor

Openpose editor for ControlNet. Full hand/face support.
MIT License
633 stars 62 forks source link

[Bug]: Unable to edit face points #60

Open Exerionius opened 8 months ago

Exerionius commented 8 months ago

Is there an existing issue for this?

What happened?

When trying to edit a pose all face points disappear from the UI. Sending the pose back into ControlNet sends it without any face points.

For example I have this pose loaded as json: image

After pressing edit I got this: image

After sending it back to ControlNet: image

Steps to reproduce the problem

  1. Go to ControlNet
  2. Enable unit 0
  3. Set Control Type to OpenPose
  4. Click Upload JSON under Image section
  5. Upload attached json pose
  6. Click Edit under Preprocessor preview section
  7. Observe how all face points dissapear

Pose file attached pose (5).json

What should have happened?

Face points should not disappear, it should be possible to edit face points.

Commit where the problem happens

webui: https://github.com/AUTOMATIC1111/stable-diffusion-webui/commit/4afaaf8a020c1df457bcf7250cb1c7f609699fa7 controlnet: https://github.com/Mikubill/sd-webui-controlnet/commit/01e4574d8e01fa00628fdae0c8215283a1c36a8d openpose-editor: https://github.com/huchenlei/sd-webui-openpose-editor/commit/cebe13e092fa3c5d810f4afbd7a45987ba8275d0

What browsers do you use to access the UI ?

Brave

Command Line Arguments

No

Console logs

2023-12-20 19:20:44 /stable-diffusion-webui
2023-12-20 19:20:44 total 812K
2023-12-20 19:20:44 drwxr-xr-x 1 root root 4.0K Dec 20 16:20 .
2023-12-20 19:20:44 drwxr-xr-x 1 root root 4.0K Dec 13 13:26 ..
2023-12-20 19:20:44 -rw-r--r-- 1 root root   48 Dec 13 13:16 .eslintignore
2023-12-20 19:20:44 -rw-r--r-- 1 root root 3.4K Dec 13 13:16 .eslintrc.js
2023-12-20 19:20:44 drwxr-xr-x 8 root root 4.0K Dec 13 13:16 .git
2023-12-20 19:20:44 -rw-r--r-- 1 root root   55 Dec 13 13:16 .git-blame-ignore-revs
2023-12-20 19:20:44 drwxr-xr-x 4 root root 4.0K Dec 13 13:16 .github
2023-12-20 19:20:44 -rw-r--r-- 1 root root  495 Dec 13 13:16 .gitignore
2023-12-20 19:20:44 -rw-r--r-- 1 root root  119 Dec 13 13:16 .pylintrc
2023-12-20 19:20:44 -rw-r--r-- 1 root root  33K Dec 13 13:16 CHANGELOG.md
2023-12-20 19:20:44 -rw-r--r-- 1 root root  243 Dec 13 13:16 CITATION.cff
2023-12-20 19:20:44 -rw-r--r-- 1 root root  657 Dec 13 13:16 CODEOWNERS
2023-12-20 19:20:44 -rw-r--r-- 1 root root  35K Dec 13 13:16 LICENSE.txt
2023-12-20 19:20:44 -rw-r--r-- 1 root root  12K Dec 13 13:16 README.md
2023-12-20 19:20:44 drwxr-xr-x 2 root root 4.0K Dec 13 13:26 __pycache__
2023-12-20 19:20:44 -rw-r--r-- 1 root root 104K Dec 20 15:35 cache.json
2023-12-20 19:20:44 lrwxrwxrwx 1 root root   29 Dec 20 16:20 config.json -> /data/config/auto/config.json
2023-12-20 19:20:44 lrwxrwxrwx 1 root root   31 Dec 20 16:20 config_states -> /data/config/auto/config_states
2023-12-20 19:20:44 drwxr-xr-x 2 root root 4.0K Dec 13 13:16 configs
2023-12-20 19:20:44 lrwxrwxrwx 1 root root   16 Dec 20 16:20 embeddings -> /data/embeddings
2023-12-20 19:20:44 -rw-r--r-- 1 root root  167 Dec 13 13:16 environment-wsl2.yaml
2023-12-20 19:20:44 lrwxrwxrwx 1 root root   28 Dec 20 16:20 extensions -> /data/config/auto/extensions
2023-12-20 19:20:44 drwxr-xr-x 1 root root 4.0K Dec 13 13:16 extensions-builtin
2023-12-20 19:20:44 drwxr-xr-x 2 root root 4.0K Dec 13 13:16 html
2023-12-20 19:20:44 drwxr-xr-x 2 root root 4.0K Dec 13 13:17 interrogate
2023-12-20 19:20:44 drwxr-xr-x 2 root root 4.0K Dec 13 13:16 javascript
2023-12-20 19:20:44 -rw-r--r-- 1 root root 1.3K Dec 13 13:16 launch.py
2023-12-20 19:20:44 drwxr-xr-x 2 root root 4.0K Dec 13 13:16 localizations
2023-12-20 19:20:44 lrwxrwxrwx 1 root root   12 Dec 20 16:20 models -> /data/models
2023-12-20 19:20:44 drwxr-xr-x 1 root root 4.0K Dec 13 13:26 modules
2023-12-20 19:20:44 drwxr-xr-x 2 root root 4.0K Dec 14 12:02 onnx_tmp
2023-12-20 19:20:44 -rw-r--r-- 1 root root  185 Dec 13 13:16 package.json
2023-12-20 19:20:44 -rw-r--r-- 1 root root  740 Dec 20 15:31 params.txt
2023-12-20 19:20:44 -rw-r--r-- 1 root root  763 Dec 13 13:16 pyproject.toml
2023-12-20 19:20:44 drwxr-xr-x 1 root root 4.0K Dec 13 13:10 repositories
2023-12-20 19:20:44 -rw-r--r-- 1 root root   49 Dec 13 13:16 requirements-test.txt
2023-12-20 19:20:44 -rw-r--r-- 1 root root  360 Dec 13 13:16 requirements.txt
2023-12-20 19:20:44 -rw-r--r-- 1 root root  556 Dec 13 13:16 requirements_versions.txt
2023-12-20 19:20:44 -rw-r--r-- 1 root root 411K Dec 13 13:16 screenshot.png
2023-12-20 19:20:44 -rw-r--r-- 1 root root 4.7K Dec 13 13:16 script.js
2023-12-20 19:20:44 drwxr-xr-x 1 root root 4.0K Dec 13 13:26 scripts
2023-12-20 19:20:44 -rw-r--r-- 1 root root  21K Dec 13 13:16 style.css
2023-12-20 19:20:44 lrwxrwxrwx 1 root root   28 Dec 20 16:20 styles.csv -> /data/config/auto/styles.csv
2023-12-20 19:20:44 drwxr-xr-x 3 root root 4.0K Dec 13 13:16 test
2023-12-20 19:20:44 drwxr-xr-x 2 root root 4.0K Dec 13 13:16 textual_inversion_templates
2023-12-20 19:20:44 drwxr-xr-x 2 root root 4.0K Dec 20 14:58 tmp
2023-12-20 19:20:44 lrwxrwxrwx 1 root root   32 Dec 20 16:20 ui-config.json -> /data/config/auto/ui-config.json
2023-12-20 19:20:44 -rw-r--r-- 1 root root  670 Dec 13 13:16 webui-macos-env.sh
2023-12-20 19:20:44 -rw-r--r-- 1 root root   84 Dec 13 13:16 webui-user.bat
2023-12-20 19:20:44 -rw-r--r-- 1 root root 1.4K Dec 13 13:16 webui-user.sh
2023-12-20 19:20:44 -rw-r--r-- 1 root root 2.2K Dec 13 13:16 webui.bat
2023-12-20 19:20:44 -rw-r--r-- 1 root root 5.4K Dec 13 13:16 webui.py
2023-12-20 19:20:44 -rwxr-xr-x 1 root root 8.0K Dec 13 13:16 webui.sh
2023-12-20 19:20:44 Mounted embeddings
2023-12-20 19:20:44 Mounted .cache
2023-12-20 19:20:44 Mounted styles.csv
2023-12-20 19:20:44 Mounted ui-config.json
2023-12-20 19:20:44 Mounted models
2023-12-20 19:20:44 Mounted .cache
2023-12-20 19:20:44 Mounted config_states
2023-12-20 19:20:44 Mounted config.json
2023-12-20 19:20:44 Mounted extensions
2023-12-20 19:20:44 Installing extension dependencies (if any)
2023-12-20 19:20:45 Skipping disabled extension (Stable-Diffusion-WebUI-TensorRT)
2023-12-20 19:20:45 /stable-diffusion-webui/./extensions/sd-webui-controlnet/install.py:3: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
2023-12-20 19:20:45   import pkg_resources
2023-12-20 19:20:52 ControlNet preprocessor location: /data/config/auto/extensions/sd-webui-controlnet/annotator/downloads
2023-12-20 19:20:52 2023-12-20 16:20:52,702 - ControlNet - INFO - ControlNet v1.1.423
2023-12-20 19:20:53 2023-12-20 16:20:53,147 - ControlNet - INFO - ControlNet v1.1.423
2023-12-20 19:20:53 Loading weights [5dd56bfa12] from /stable-diffusion-webui/models/Stable-diffusion/flat2DAnimerge_v30.safetensors
2023-12-20 19:20:54 Running on local URL:  http://0.0.0.0:7860
2023-12-20 19:20:55 
2023-12-20 19:20:55 To create a public link, set `share=True` in `launch()`.
2023-12-20 19:20:55 Startup time: 9.4s (import torch: 2.2s, import gradio: 0.9s, setup paths: 1.0s, initialize shared: 0.4s, other imports: 0.5s, load scripts: 2.7s, create ui: 0.9s, gradio launch: 0.3s).
2023-12-20 19:20:56 Creating model from config: /stable-diffusion-webui/configs/v1-inference.yaml
2023-12-20 19:21:16 Loading VAE weights specified in settings: /stable-diffusion-webui/models/VAE/vae-ft-mse-840000-ema-pruned.ckpt
2023-12-20 19:21:18 Applying attention optimization: xformers... done.
2023-12-20 19:21:18 Model loaded in 25.2s (load weights from disk: 2.6s, create model: 0.4s, apply weights to model: 20.1s, load VAE: 1.4s, calculate empty prompt: 0.7s).
2023-12-20 19:22:37 2023-12-20 16:22:37,705 - ControlNet - INFO - Preview as input is enabled.

Browser logs

content.js:57 

       Failed to load resource: net::ERR_FILE_NOT_FOUND
ns @ content.js:57
Show 1 more frame
Show less
content.js:57 content: unable to decode unknown
ns @ content.js:57
index-52429a0e.js:584 Wrong number of keypoints for openpose face. Expect 70 but got 69.
create @ index-52429a0e.js:584
index-52429a0e.js:584 Wrong number of keypoints for openpose face. Expect 70 but got 69.
create @ index-52429a0e.js:584

Additional information

No response

huchenlei commented 8 months ago

How did you obtain the openpose file? ControlNet preprocessors should always produce 70 keypoints for face.

huchenlei commented 8 months ago

Maybe I should remove the 70 keypoints constraint for face as these keypoints are not connected?

Exerionius commented 8 months ago

How did you obtain the openpose file? ControlNet preprocessors should always produce 70 keypoints for face.

Turns out they don't.

Here's how to get a broken json:

  1. Enable a ControlNet Unit
  2. Select OpenPose as Control Type
  3. Select openpose_full as Preprocessor
  4. Upload the attached image
  5. Click Run preprocessor
  6. Resulting json is attached image

result.json image used