search

ThereforeGames / unprompted

Templating language written for Stable Diffusion workflows. Available as an extension for the Automatic1111 WebUI.
778 stars 66 forks source link

Bodysnatcher + Adetailer: Pillow fails when inverting txt2mask: OSError: not supported for this image mode #229

Open slavasav opened 10 months ago

slavasav commented 10 months ago

Due diligence

Describe the bug

Using the bodysnatcher template with fidelity_v1 in img2img, it fails to create the mask with exception in Pillow: OSError: not supported for this image mode. Using Adetailer with face detection, a prompt and "Inpaint only masked" mode. If I disable ADetailer - there is no exception. Looking through Pillow issues and latest fixed it does not seem that Pillow is the problem. Unprompted extension folder is renamed to _unpropmpted, so should go first.

The usecase is to use bodysnatcher and then replace the face with a more detailed lora using ADetailer prompt.

Also take note of the WARNING - [normal_map.processor_res] Invalid value(-1), using default value 512.

Prompt

[call "common\functions\bodysnatcher" prefix=photo subject=woman simple_description="best light, beauty" class=woman interrogate=1 max_image_size=0 inference_preset=general_v2 body_controlnet_preset=fidelity_v1 face_controlnet_preset=none keep_hands=1 keep_feet=1 mask_method=clipseg manual_mask_mode=subtract mask_informs_size=1 mask_size_limit=768.0 background_mode=0 bypass_zoom_enhance=1 zoom_enhance_base_cfg=10 zoom_enhance_inherit_negative=1 show_original=0 color_correct_method=hm-mkl-hm color_correct_timing=pre color_correct_strength=1 debug=0 name='Bodysnatcher v1.5.0']

Log output

2024-01-06 19:09:02,636  (INFO)     [Unprompted.txt2mask] Using cached clipseg model.██| 18/18 [00:24<00:00,  2.12it/s]
2024-01-06 19:09:03,339 - ControlNet - INFO - unit_separate = False, style_align = False
2024-01-06 19:09:03,339 - ControlNet - WARNING - [depth.processor_res] Invalid value(-1), using default value 512.
2024-01-06 19:09:03,341 - ControlNet - INFO - Loading model: control_v11f1p_sd15_depth
2024-01-06 19:09:03,656 - ControlNet - INFO - Loading model: control_v11f1p_sd15_depth [cfd03158]
2024-01-06 19:09:04,409 - ControlNet - INFO - Loaded state_dict from [C:\Users\X\stable-diffusion-webui\extensions\sd-webui-controlnet\models\control_v11f1p_sd15_depth.pth]
2024-01-06 19:09:04,409 - ControlNet - INFO - controlnet_default_config
2024-01-06 19:09:07,317 - ControlNet - INFO - ControlNet model control_v11f1p_sd15_depth [cfd03158] loaded.
2024-01-06 19:09:07,453 - ControlNet - INFO - Loading preprocessor: depth
2024-01-06 19:09:07,453 - ControlNet - INFO - preprocessor resolution = 512
2024-01-06 19:09:07,493 - ControlNet - WARNING - [normal_map.processor_res] Invalid value(-1), using default value 512.
2024-01-06 19:09:07,494 - ControlNet - WARNING - [normal_map.threshold_a] Invalid value(-1), using default value 0.4.
2024-01-06 19:09:07,495 - ControlNet - INFO - Loading model: control_v11p_sd15_normalbae_fp16
2024-01-06 19:09:07,791 - ControlNet - INFO - Loading model: control_v11p_sd15_normalbae_fp16 [592a19d8]
2024-01-06 19:09:07,821 - ControlNet - INFO - Loaded state_dict from [C:\Users\X\stable-diffusion-webui\extensions\sd-webui-controlnet\models\control_v11p_sd15_normalbae_fp16.safetensors]
2024-01-06 19:09:07,821 - ControlNet - INFO - controlnet_default_config
2024-01-06 19:09:10,636 - ControlNet - INFO - ControlNet model control_v11p_sd15_normalbae_fp16 [592a19d8] loaded.
2024-01-06 19:09:10,711 - ControlNet - INFO - Loading preprocessor: normal_map
2024-01-06 19:09:10,711 - ControlNet - INFO - preprocessor resolution = 512
2024-01-06 19:09:10,766 - ControlNet - WARNING - [reference_adain.threshold_a] Invalid value(-1), using default value 0.5.
2024-01-06 19:09:10,782 - ControlNet - INFO - Loading preprocessor: reference_adain
2024-01-06 19:09:10,783 - ControlNet - INFO - preprocessor resolution = 512
2024-01-06 19:09:10,816 - ControlNet - INFO - Loading model: control_v11p_sd15s2_lineart_anime_fp16
2024-01-06 19:09:11,103 - ControlNet - INFO - Loading model: control_v11p_sd15s2_lineart_anime_fp16 [c58f338b]
2024-01-06 19:09:11,134 - ControlNet - INFO - Loaded state_dict from [C:\Users\X\stable-diffusion-webui\extensions\sd-webui-controlnet\models\control_v11p_sd15s2_lineart_anime_fp16.safetensors]
2024-01-06 19:09:11,134 - ControlNet - INFO - controlnet_default_config
2024-01-06 19:09:14,054 - ControlNet - INFO - ControlNet model control_v11p_sd15s2_lineart_anime_fp16 [c58f338b] loaded.2024-01-06 19:09:14,120 - ControlNet - INFO - Loading preprocessor: lineart_anime_denoise
2024-01-06 19:09:14,120 - ControlNet - INFO - preprocessor resolution = 512
2024-01-06 19:09:14,184 - ControlNet - INFO - ControlNet Hooked - Time = 10.850613117218018
  0%|                                                                                           | 0/18 [00:00<?, ?it/s]2024-01-06 19:09:14,928 - ControlNet - INFO - ControlNet used torch.float16 VAE to encode torch.Size([1, 4, 64, 96]).
100%|██████████████████████████████████████████████████████████████████████████████████| 18/18 [00:10<00:00,  1.78it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 18/18 [00:08<00:00,  2.08it/s]
0: 448x640 1 face, 12.0ms
Speed: 3.0ms preprocess, 12.0ms inference, 2.0ms postprocess per image at shape (1, 3, 448, 640)
100%|████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:02<00:00,  3.96it/s]
2024-01-06 19:09:31,813  (INFO)     [Unprompted.txt2mask] Using cached clipseg model.
*** Error running process: C:\Users\X\stable-diffusion-webui\extensions\_unprompted\scripts\unprompted.py
    Traceback (most recent call last):
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shortcodes.py", line 140, in render
        return str(self.handler(self.token.keyword, self.pargs, self.kwargs, context, content))
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shared.py", line 93, in handler
        return (self.shortcode_objects[f"{keyword}"].run_block(pargs, kwargs, context, content))
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted/shortcodes\stable_diffusion\txt2mask.py", line 612, in run_block
        self.image_mask = get_mask().resize((self.init_image.width, self.init_image.height))
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted/shortcodes\stable_diffusion\txt2mask.py", line 510, in get_mask
        self.Unprompted.shortcode_user_vars["image_mask"] = ImageOps.invert(self.Unprompted.shortcode_user_vars["image_mask"])
      File "C:\Users\X\stable-diffusion-webui\venv\lib\site-packages\PIL\ImageOps.py", line 534, in invert
        return image.point(lut) if image.mode == "1" else _lut(image, lut)
      File "C:\Users\X\stable-diffusion-webui\venv\lib\site-packages\PIL\ImageOps.py", line 60, in _lut
        raise OSError(msg)
    OSError: not supported for this image mode

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shortcodes.py", line 140, in render
        return str(self.handler(self.token.keyword, self.pargs, self.kwargs, context, content))
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shared.py", line 85, in handler
        return (self.shortcode_objects[f"{keyword}"].run_block(pargs, kwargs, context, content))
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted/shortcodes\basic\if.py", line 55, in run_block
        to_return = self.Unprompted.process_string(content, context)
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shared.py", line 212, in process_string
        string = self.shortcode_parser.parse(self.sanitize_pre(string, self.Config.syntax.sanitize_before), context)
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shortcodes.py", line 251, in parse
        return stack.pop().render(context).replace(self.esc_start,"")
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shortcodes.py", line 55, in render
        return ''.join(child.render(context) for child in self.children)
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shortcodes.py", line 55, in <genexpr>
        return ''.join(child.render(context) for child in self.children)
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shortcodes.py", line 144, in render
        raise ShortcodeRenderingError(msg) from ex
    lib_unprompted.shortcodes.ShortcodeRenderingError: An exception was raised while rendering the 'txt2mask' shortcode in line 1.

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shortcodes.py", line 140, in render
        return str(self.handler(self.token.keyword, self.pargs, self.kwargs, context, content))
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shared.py", line 85, in handler
        return (self.shortcode_objects[f"{keyword}"].run_block(pargs, kwargs, context, content))
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted/shortcodes\basic\if.py", line 55, in run_block
        to_return = self.Unprompted.process_string(content, context)
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shared.py", line 212, in process_string
        string = self.shortcode_parser.parse(self.sanitize_pre(string, self.Config.syntax.sanitize_before), context)
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shortcodes.py", line 251, in parse
        return stack.pop().render(context).replace(self.esc_start,"")
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shortcodes.py", line 55, in render
        return ''.join(child.render(context) for child in self.children)
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shortcodes.py", line 55, in <genexpr>
        return ''.join(child.render(context) for child in self.children)
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shortcodes.py", line 144, in render
        raise ShortcodeRenderingError(msg) from ex
    lib_unprompted.shortcodes.ShortcodeRenderingError: An exception was raised while rendering the 'if' shortcode in line 1.

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shortcodes.py", line 115, in render
        return str(self.handler(self.token.keyword, self.pargs, self.kwargs, context))
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shared.py", line 74, in handler
        return (self.shortcode_objects[f"{keyword}"].run_atomic(pargs, kwargs, context))
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted/shortcodes\basic\call.py", line 66, in run_atomic
        contents = self.Unprompted.process_string(contents, next_context)
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shared.py", line 212, in process_string
        string = self.shortcode_parser.parse(self.sanitize_pre(string, self.Config.syntax.sanitize_before), context)
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shortcodes.py", line 251, in parse
        return stack.pop().render(context).replace(self.esc_start,"")
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shortcodes.py", line 55, in render
        return ''.join(child.render(context) for child in self.children)
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shortcodes.py", line 55, in <genexpr>
        return ''.join(child.render(context) for child in self.children)
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shortcodes.py", line 144, in render
        raise ShortcodeRenderingError(msg) from ex
    lib_unprompted.shortcodes.ShortcodeRenderingError: An exception was raised while rendering the 'if' shortcode in line 1.

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "C:\Users\X\stable-diffusion-webui\modules\scripts.py", line 718, in process
        script.process(p, *script_args)
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\scripts\unprompted.py", line 748, in process
        prompt_result = Unprompted.start(apply_prompt_template(Unprompted.original_prompt, Unprompted.Config.templates.default))
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shared.py", line 185, in start
        result = self.process_string(string)
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shared.py", line 212, in process_string
        string = self.shortcode_parser.parse(self.sanitize_pre(string, self.Config.syntax.sanitize_before), context)
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shortcodes.py", line 251, in parse
        return stack.pop().render(context).replace(self.esc_start,"")
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shortcodes.py", line 55, in render
        return ''.join(child.render(context) for child in self.children)
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shortcodes.py", line 55, in <genexpr>
        return ''.join(child.render(context) for child in self.children)
      File "C:\Users\X\stable-diffusion-webui\extensions\_unprompted\lib_unprompted\shortcodes.py", line 119, in render
        raise ShortcodeRenderingError(msg) from ex
    lib_unprompted.shortcodes.ShortcodeRenderingError: An exception was raised while rendering the 'call' shortcode in line 1.

---
2024-01-06 19:09:32,067 - ControlNet - INFO - unit_separate = False, style_align = False
2024-01-06 19:09:32,067 - ControlNet - WARNING - [depth.processor_res] Invalid value(-1), using default value 512.
2024-01-06 19:09:32,068 - ControlNet - INFO - Loading model: control_v11f1p_sd15_depth
2024-01-06 19:09:32,361 - ControlNet - INFO - Loading model: control_v11f1p_sd15_depth [cfd03158]
2024-01-06 19:09:33,109 - ControlNet - INFO - Loaded state_dict from [C:\Users\X\stable-diffusion-webui\extensions\sd-webui-controlnet\models\control_v11f1p_sd15_depth.pth]
2024-01-06 19:09:33,110 - ControlNet - INFO - controlnet_default_config
2024-01-06 19:09:36,190 - ControlNet - INFO - ControlNet model control_v11f1p_sd15_depth [cfd03158] loaded.
2024-01-06 19:09:36,343 - ControlNet - INFO - Loading preprocessor: depth
2024-01-06 19:09:36,343 - ControlNet - INFO - preprocessor resolution = 512
2024-01-06 19:09:36,380 - ControlNet - WARNING - [normal_map.processor_res] Invalid value(-1), using default value 512.
2024-01-06 19:09:36,380 - ControlNet - WARNING - [normal_map.threshold_a] Invalid value(-1), using default value 0.4.
2024-01-06 19:09:36,381 - ControlNet - INFO - Loading model: control_v11p_sd15_normalbae_fp16
2024-01-06 19:09:36,704 - ControlNet - INFO - Loading model: control_v11p_sd15_normalbae_fp16 [592a19d8]
2024-01-06 19:09:36,738 - ControlNet - INFO - Loaded state_dict from [C:\Users\X\stable-diffusion-webui\extensions\sd-webui-controlnet\models\control_v11p_sd15_normalbae_fp16.safetensors]
2024-01-06 19:09:36,739 - ControlNet - INFO - controlnet_default_config
2024-01-06 19:09:39,870 - ControlNet - INFO - ControlNet model control_v11p_sd15_normalbae_fp16 [592a19d8] loaded.
2024-01-06 19:09:39,952 - ControlNet - INFO - Loading preprocessor: normal_map
2024-01-06 19:09:39,952 - ControlNet - INFO - preprocessor resolution = 512
2024-01-06 19:09:39,996 - ControlNet - WARNING - [reference_adain.threshold_a] Invalid value(-1), using default value 0.5.
2024-01-06 19:09:40,013 - ControlNet - INFO - Loading preprocessor: reference_adain
2024-01-06 19:09:40,013 - ControlNet - INFO - preprocessor resolution = 512
2024-01-06 19:09:40,046 - ControlNet - INFO - Loading model: control_v11p_sd15s2_lineart_anime_fp16
2024-01-06 19:09:40,322 - ControlNet - INFO - Loading model: control_v11p_sd15s2_lineart_anime_fp16 [c58f338b]
2024-01-06 19:09:40,362 - ControlNet - INFO - Loaded state_dict from [C:\Users\X\stable-diffusion-webui\extensions\sd-webui-controlnet\models\control_v11p_sd15s2_lineart_anime_fp16.safetensors]
2024-01-06 19:09:40,362 - ControlNet - INFO - controlnet_default_config
2024-01-06 19:09:43,288 - ControlNet - INFO - ControlNet model control_v11p_sd15s2_lineart_anime_fp16 [c58f338b] loaded.2024-01-06 19:09:43,364 - ControlNet - INFO - Loading preprocessor: lineart_anime_denoise
2024-01-06 19:09:43,364 - ControlNet - INFO - preprocessor resolution = 512
2024-01-06 19:09:43,423 - ControlNet - INFO - ControlNet Hooked - Time = 11.361028671264648

Unprompted version

v10.6.0

WebUI version

v1.7.0

Other comments

No response

slavasav commented 10 months ago

I must add that ADetailer still managers to fix the face, so not sure if the exception breaks anything in the flow.

slavasav commented 10 months ago

I don't think it relates to ADetailer at all now, I think this line is the culprit: https://github.com/ThereforeGames/unprompted/blob/2d4a184597791875054bfc4fa65def296e789c9b/shortcodes/stable_diffusion/txt2mask.py#L498

It keeps the previous mask and when the size has changed it cannot be inverted. The fix would be to clear the current image_mask on next txt2mask call.