Mikubill / sd-webui-controlnet

WebUI extension for ControlNet
GNU General Public License v3.0
17.07k stars 1.96k forks source link

[Feature Request] Postprocess scripts #1674

Closed olamedia closed 12 months ago

olamedia commented 1 year ago

What problem the feature is trying to solve: Postprocess generated maps. For example, script that loads openpose json, and trying to fix pose using various models.

Expected workflow for user:

  1. select script from selectbox, similar to scripts at the bottom of the txt2img and img2img pages

Expected workflow for script:

  1. maps are generated
  2. receive list of generated maps
  3. postprocess them
  4. output list of processed maps
huchenlei commented 1 year ago

For openpose, if you have a script that can improve detection accuracy, I would suggest directly add that to ControlNet's openpose detection preprocessor, instead of doing a post process script.

Also, for now if you really want to correct the inaccurate detections in openpose, we have the integrated openpose editor to do that job: https://github.com/huchenlei/sd-webui-openpose-editor

olamedia commented 1 year ago

It's to modify already generated maps. For openpose it should be possible to calculate and fix impossible angles and proportions (for specific body type) automatically.

I like this editor, but script would be able to perform this in batch.

olamedia commented 1 year ago

An example of postprocessing for openpose can be opensim which can "constraining angles to a biomechanically accurate model" - a conversion to the opensim is already exists as https://github.com/perfanalytics/pose2sim#with-openpose

olamedia commented 1 year ago

Here is another example of correction script https://github.com/DGS-Korpus/Public-Corpus-OpenPose-error-corrector