search

Bing-su / adetailer

Auto detecting, masking and inpainting with detection model.
GNU Affero General Public License v3.0
4.17k stars 323 forks source link

[Bug]: ADetailer breaks when sd_webui_SAG is installed. #208

Closed lendrick closed 1 year ago

lendrick commented 1 year ago

Describe the bug

adetailer causes an error when sd_webui_SAG is installed. This appears to have started happening some time after adetailer commit 0f3b6b0.

The entire console log, including python version information, webui version, commit hash, commandline arguments and error traces.

(sdweb) PS C:\ai\automatic> webui.bat
Using VENV: C:\ai\automatic\venv
18:40:56-390780 INFO     Starting SD.Next
18:40:56-400789 INFO     Python 3.10.11 on Windows
18:40:56-444828 INFO     Version: fb274d3d Tue Jun 13 00:50:34 2023 -0400
18:40:56-824210 INFO     Latest published version: 484b1166640ef1560d4725e39c60c62f98df293d 2023-07-14T21:20:00Z
18:40:56-830215 INFO     nVidia CUDA toolkit detected
18:41:02-436449 INFO     Torch 2.0.0+cu118
18:41:02-471478 INFO     Torch backend: nVidia CUDA 11.8 cuDNN 8800
18:41:02-474481 INFO     Torch detected GPU: NVIDIA GeForce RTX 4090 VRAM 24564 Arch (8, 9) Cores 128
18:41:02-655898 WARNING  Modified files: ['modules/lora', 'modules/lycoris', 'modules/sd_samplers_kdiffusion.py',
                         'modules/sd_vae.py', 'webui.py', 'metadata.json.old', 'modules.old/',
                         'scripts/loopback_scaler.py', 'scripts/lora_weight_randomize.py', 'shap_e_model_cache/']
18:41:02-746052 INFO     Disabled extensions-builtin: ['sd_civitai_extension',
                         'a1111-stable-diffusion-webui-vram-estimator', 'OneButtonPrompt', 'sd-canvas-editor',
                         'canvas-zoom', 'PBRemTools', 'sd-pixel', 'sd-webui-lua', 'sd-webui-additional-networks',
                         'Stable-Diffusion-Webui-Civitai-Helper', 'sd-webui-stablesr', 'sd-model-preview-xd',
                         'weight_gradient', 'stable-diffusion-webui-model-toolkit', 'sd-webui-llul',
                         'sd-webui-3d-open-pose-editor', 'stable-diffusion-webui-two-shot',
                         'stable-diffusion-webui-composable-lora']
18:41:02-754059 INFO     Enabled extensions-builtin: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR',
                         'Lora', 'multidiffusion-upscaler-for-automatic1111', 'ScuNET', 'sd-dynamic-thresholding',
                         'sd-extension-system-info', 'sd-webui-agent-scheduler', 'sd-webui-controlnet',
                         'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR']
18:41:02-771074 INFO     Disabled extensions: ['sd_civitai_extension', 'a1111-stable-diffusion-webui-vram-estimator',
                         'OneButtonPrompt', 'sd-canvas-editor', 'canvas-zoom', 'PBRemTools', 'sd-pixel',
                         'sd-webui-lua', 'sd-webui-additional-networks', 'Stable-Diffusion-Webui-Civitai-Helper',
                         'sd-webui-stablesr', 'sd-model-preview-xd', 'weight_gradient',
                         'stable-diffusion-webui-model-toolkit', 'sd-webui-llul', 'sd-webui-3d-open-pose-editor',
                         'stable-diffusion-webui-two-shot', 'stable-diffusion-webui-composable-lora']
18:41:02-779082 INFO     Enabled extensions: ['a1111-mini-paint', 'a1111-sd-webui-haku-img',
                         'a1111-sd-webui-tagcomplete', 'adetailer', 'CFG-Schedule-for-Automatic1111-SD',
                         'Config-Presets', 'model_preset_manager', 'prompt-fusion-extension', 'sd-dynamic-prompts',
                         'sd-webui-aspect-ratio-helper', 'sd-webui-infinite-image-browsing',
                         'sd-webui-inpaint-anything', 'sd-webui-lora-block-weight', 'sd-webui-model-converter',
                         'sd-webui-openpose-editor', 'sd-webui-segment-anything', 'sd-webui-supermerger',
                         'sd-webui-txt-img-to-3d-model', 'sd_webui_SAG', 'sd_web_ui_preset_utils',
                         'stable-diffusion-webui-dataset-tag-editor', 'stable-diffusion-webui-state',
                         'ultimate-upscale-for-automatic1111']
18:41:02-827169 INFO     Verifying requirements
18:41:02-851697 INFO     Verifying packages
18:41:02-853934 INFO     Installing repositories
18:41:05-527037 INFO     Installing submodules
18:41:08-081694 INFO     Updating submodules
18:42:06-265805 INFO     Extension installed packages: stable-diffusion-webui-images-browser ['pyarrow==12.0.1',
                         'xxhash==3.2.0', 'datasets==2.13.1']
18:42:27-053562 INFO     Extension installed packages: sd-dynamic-prompts ['dynamicprompts==0.28.0']
18:43:07-477551 INFO     Extensions enabled: ['a1111-sd-webui-lycoris', 'clip-interrogator-ext', 'LDSR', 'Lora',
                         'multidiffusion-upscaler-for-automatic1111', 'ScuNET', 'sd-dynamic-thresholding',
                         'sd-extension-system-info', 'sd-webui-agent-scheduler', 'sd-webui-controlnet',
                         'stable-diffusion-webui-images-browser', 'stable-diffusion-webui-rembg', 'SwinIR',
                         'a1111-mini-paint', 'a1111-sd-webui-haku-img', 'a1111-sd-webui-tagcomplete', 'adetailer',
                         'CFG-Schedule-for-Automatic1111-SD', 'Config-Presets', 'model_preset_manager',
                         'prompt-fusion-extension', 'sd-dynamic-prompts', 'sd-webui-aspect-ratio-helper',
                         'sd-webui-infinite-image-browsing', 'sd-webui-inpaint-anything', 'sd-webui-lora-block-weight',
                         'sd-webui-model-converter', 'sd-webui-openpose-editor', 'sd-webui-segment-anything',
                         'sd-webui-supermerger', 'sd-webui-txt-img-to-3d-model', 'sd_webui_SAG',
                         'sd_web_ui_preset_utils', 'stable-diffusion-webui-dataset-tag-editor',
                         'stable-diffusion-webui-state', 'ultimate-upscale-for-automatic1111']
18:43:07-489584 INFO     Updating Wiki
18:43:08-728923 INFO     Extension preload: 0.0s C:\ai\automatic\extensions-builtin
18:43:08-733928 INFO     Extension preload: 0.0s C:\ai\automatic\extensions
18:43:08-768143 INFO     Server arguments: []
18:43:19-461701 INFO     Libraries loaded
18:43:19-465704 INFO     Using data path: C:\ai\automatic
18:43:19-478736 INFO     Available VAEs: C:\ai\automatic\models\VAE 12
18:43:25-123629 INFO     Available models: C:\ai\automatic\models\Stable-diffusion 263
18:43:29-864321 INFO     ControlNet v1.1.232
ControlNet preprocessor location: C:\ai\automatic\extensions-builtin\sd-webui-controlnet\annotator\downloads
18:43:30-566443 INFO     ControlNet v1.1.232
[-] ADetailer initialized. version: 23.7.5, num models: 9
18:43:45-928096 INFO     Loading UI theme: name=black-orange style=Auto
add tab
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
18:44:02-191108 INFO     Local URL: http://127.0.0.1:7860/
18:44:02-193109 INFO     Initializing middleware
18:44:02-516141 INFO     [AgentScheduler] Task queue is empty
18:44:02-519144 INFO     [AgentScheduler] Registering APIs
18:44:04-016577 Loading weights: INFO    C:\ai\automatic\models\Stable-diffusion\!EnvyMix_v12.safetensors  Startup time: ------------------55. 2s 0.0/9.2 GB( torch-:--:--=4.7s gradio=2.5s libraries=3.4s models=5.6s codeformer=0.1s
                         scripts=20.5s onchange=0.2s ui=13.3s launch=2.9s app-started=0.6s checkpoint=1.2s)
Loading weights: C:\ai\automatic\models\Stable-diffusion\!EnvyMix_v12.safetensors ------------------ 0.0/9.2 GB -:--:--
18:44:05-297335 INFO     Torch override VAE dtype: no-half set
18:44:05-299337 INFO     Setting Torch parameters: dtype=torch.float32 vae=torch.float32 unet=torch.float32
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Loading weights: C:\ai\automatic\models\VAE\clearvae_main.safetensors ---------------------------- 0.0/334.6 MB -:--:--
18:44:24-885386 INFO     Applying xformers cross attention optimization
18:44:27-727720 INFO     Embeddings: loaded=51 skipped=24
18:44:27-759758 INFO     Model loaded in 24.3s (load=1.9s create=0.5s apply=14.7s vae=2.4s move=2.0s hijack=0.1s
                         embeddings=2.8s)
18:44:29-935470 INFO     Model load finished: {'ram': {'used': 4.75, 'total': 15.93}, 'gpu': {'used': 3.78, 'total':
                         23.99}, 'retries': 0, 'oom': 0}
100%|██████████████████████████████████████████████████████████████████████████████████| 84/84 [00:13<00:00,  6.21it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 29/29 [00:11<00:00,  2.59it/s]
18:49:32-594333 INFO
18:49:33-325931 INFO     0: 640x448 1 face, 318.5ms
18:49:33-327933 INFO     Speed: 8.0ms preprocess, 318.5ms inference, 45.1ms postprocess per image at shape (1, 3, 640,
                         640)
  0%|                                                                                           | 0/19 [00:00<?, ?it/s]18:49:34-609499 ERROR    executing callback: C:\ai\automatic\extensions\sd_webui_SAG\scripts\SAG.py
                         cfg_denoised_callback: RuntimeError
┌───────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────┐
│ C:\ai\automatic\modules\script_callbacks.py:226 in cfg_denoised_callback                                            │
│                                                                                                                     │
│   225 │   │   │   t0 = time.time()                                                                                  │
│ > 226 │   │   │   c.callback(params)                                                                                │
│   227 │   │   │   timer(t0, c.script, 'cfg_denoised')                                                               │
│                                                                                                                     │
│ C:\ai\automatic\extensions\sd_webui_SAG\scripts\SAG.py:214 in denoised_callback                                     │
│                                                                                                                     │
│   213 │   │   attn_mask = (                                                                                         │
│ > 214 │   │   │   attn_mask.reshape(b, middle_layer_latent_size[0], middle_layer_latent_size[1                      │
│   215 │   │   │   .unsqueeze(1)                                                                                     │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
RuntimeError: shape '[1, 8, 8]' is invalid for input of size 384
18:49:34-999651 ERROR    executing callback: C:\ai\automatic\extensions\sd_webui_SAG\scripts\SAG.py
                         cfg_after_cfg_callback: RuntimeError
┌───────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────┐
│ C:\ai\automatic\modules\script_callbacks.py:236 in cfg_after_cfg_callback                                           │
│                                                                                                                     │
│   235 │   │   │   t0 = time.time()                                                                                  │
│ > 236 │   │   │   c.callback(params)                                                                                │
│   237 │   │   │   timer(t0, c.script, 'cfg_after_cfg')                                                              │
│                                                                                                                     │
│ C:\ai\automatic\extensions\sd_webui_SAG\scripts\SAG.py:242 in cfg_after_cfg_callback                                │
│                                                                                                                     │
│   241 │   │                                                                                                         │
│ > 242 │   │   params.x = params.x + (current_uncond_pred - (current_degraded_pred + current_de                      │
│   243 │   │   params.output_altered = True                                                                          │
└─────────────────────────────────────────────────────────────────────────────────────────────────────

[error repeats each iteration]

List of installed extensions

see console log

Bing-su commented 1 year ago

I looked at the SAG code and found 2 solutions,

  1. fix SAG.py like this

    # sd_webui_SAG/scripts/SAG.py, line 292.

def postprocess(self, p, processed, *args):
    enabled, scale, sag_mask_threshold = args
    if enabled:
        # restore original self attention module forward function
        attn_module = shared.sd_model.model.diffusion_model.middle_block._modules['1'].transformer_blocks._modules[
            '0'].attn1
        attn_module.forward = saved_original_selfattn_forward
    sag_enabled = False
    return
        sag_enabled = False   # <- add this line
        return
  1. add SAG to the script names in the ADetailer settings.

Cap 2023-07-04 09-04-28-256

Not tested. sorry.

lendrick commented 1 year ago

I can confirm that solution #2 works. Thanks!!