search

ThereforeGames / unprompted

Templating language written for Stable Diffusion workflows. Available as an extension for the Automatic1111 WebUI.
777 stars 66 forks source link

Unprompted passing wrong arguments (too many, wrong order) to img2Img #245

Closed Sporking closed 6 months ago

Sporking commented 6 months ago

Due diligence

Describe the bug

I attempted to use the prompt

a pretty woman[after][zoom_enhance debug replacement="blue face" denoising_max=0.5 contour_padding=100][/after]

on current updates of Automatic1111 and Unprompted, and got thrown exceptions and no images as output.

(Note: this same prompt was used in #190, and worked fine at that time.)

When I tried this, I noticed that only the following files were created (due to 'debug' in the prompt):

    zoom_enhance_img0_0_starting_image.png
    zoom_enhance_img0_1_mask_image.png
    zoom_enhance_img0_2_mask_with_overlay.png
    zoom_enhance_img0_3_cropped.png
    zoom_enhance_img0_4_mask_with_blur.png

    (no files past this point were generated)

I got errors in the console that looked like this:

2024-04-22 00:09:16,925  (INFO)     [Unprompted.after] Processing After content for batch 0, block 0...██████████████████████████████████████████████████████████| 20/20 [00:01<00:00,  9.69it/s]
2024-04-22 00:09:16,926  (WARNING)  [Unprompted.zoom_enhance] Since you are using the debug flag, Zoom Enhance will save test images to the root of your WebUI folder - don't forget!
2024-04-22 00:09:16,991  (INFO)     [Unprompted.txt2mask] Using cached clipseg model.
2024-04-22 00:09:17,127  (WARNING)  [Unprompted.zoom_enhance] Processing mode: Shortcode-Based (may not be compatible with ControlNet)
2024-04-22 00:09:17,127  (ERROR)    [Unprompted.img2img] Exception while running the img2img task
Traceback (most recent call last):
  File "C:\A1111 Web UI Autoinstaller\stable-diffusion-webui\extensions\unprompted/shortcodes\stable_diffusion\img2img.py", line 110, in run_atomic
    img2img_result = img2img_func(
  File "C:\A1111 Web UI Autoinstaller\stable-diffusion-webui\modules\img2img.py", line 187, in img2img
    assert 0. <= denoising_strength <= 1., 'can only work with strength in [0.0, 1.0]'
TypeError: '<=' not supported between instances of 'float' and 'NoneType'
2024-04-22 00:09:17,128  (ERROR)    [Unprompted.img2img] The returned object does not appear to contain an image: []
2024-04-22 00:09:17,128  (ERROR)    [Unprompted.zoom_enhance] Exception while running the img2img task
Traceback (most recent call last):
  File "C:\A1111 Web UI Autoinstaller\stable-diffusion-webui\extensions\unprompted/shortcodes\stable_diffusion\zoom_enhance.py", line 398, in run_atomic
    if debug: fixed_image.save(f"zoom_enhance_img{image_idx}_5_enhanced.png")
AttributeError: 'str' object has no attribute 'save'

Note that the error message above seems to be complaining that the 'denoising_strength' parameter has the value None rather than being a float in the expected range of 0..1.

I investigated and found this call to to the 'img2img' function, in C:\A1111 Web UI Autoinstaller\stable-diffusion-webui\extensions\unprompted/shortcodes\stable_diffusion\img2img.py:

            if self.Unprompted.webui == "forge":
                img2img_func = modules.img2img.img2img_function
            else:
                img2img_func = modules.img2img.img2img

            # img2img_function(id_task: str, mode: int, prompt: str, negative_prompt: str, prompt_styles, init_img, sketch, init_img_with_mask, inpaint_color_sketch, inpaint_color_sketch_orig, init_img_inpaint, init_mask_inpaint, steps: int, sampler_name: str, mask_blur: int, mask_alpha: float, inpainting_fill: int, n_iter: int, batch_size: int, cfg_scale: float, image_cfg_scale: float, denoising_strength: float, selected_scale_tab: int, height: int, width: int, scale_by: float, resize_mode: int, inpaint_full_res: bool, inpaint_full_res_padding: int, inpainting_mask_invert: int, img2img_batch_input_dir: str, img2img_batch_output_dir: str, img2img_batch_inpaint_mask_dir: str, override_settings_texts, img2img_batch_use_png_info: bool, img2img_batch_png_info_props: list, img2img_batch_png_info_dir: str, request: gr.Request, *args)

            img2img_result = img2img_func(
                "unprompted_img2img",  #id_task
                int(self.Unprompted.shortcode_user_vars["mode"]) if "mode" in self.Unprompted.shortcode_user_vars else 0,  #p.mode
                prompt,
                negative_prompt,
                [],  # prompt_styles
                image,  # init_img
                None,  # sketch
                init_img_with_mask,  # p.init_img_with_mask
                None,  # inpaint_color_sketch
                None,  # inpaint_color_sketch_orig
                image,  # p.init_img_inpaint
                init_mask,  # p.init_mask_inpaint
                self.Unprompted.shortcode_user_vars["steps"],
                self.Unprompted.shortcode_user_vars["sampler_name"],
                self.Unprompted.shortcode_user_vars["mask_blur"] if "mask_blur" in self.Unprompted.shortcode_user_vars else 0,  # p.mask_blur
                0.0,  #p.mask_alpha
                0,  # p.inpainting_fill
                1,  # n_iter
                1,  # batch_size
                self.Unprompted.shortcode_user_vars["cfg_scale"],
                self.Unprompted.shortcode_user_vars["image_cfg_scale"] if "image_cfg_scale" in self.Unprompted.shortcode_user_vars else None,
                self.Unprompted.shortcode_user_vars["denoising_strength"] if self.Unprompted.shortcode_user_vars["denoising_strength"] is not None else 1.0,
                0,  #selected_scale_tab
                current_height,  #self.Unprompted.shortcode_user_vars["height"],
                current_width,  #self.Unprompted.shortcode_user_vars["width"],
                1.0,  #scale_by
                self.Unprompted.shortcode_user_vars["resize_mode"] if "resize_mode" in self.Unprompted.shortcode_user_vars else 1,
                self.Unprompted.shortcode_user_vars["inpaint_full_res"] if "inpaint_full_res" in self.Unprompted.shortcode_user_vars else True,  # p.inpaint_full_res
                self.Unprompted.shortcode_user_vars["inpaint_full_res_padding"] if "inpaint_full_res_padding" in self.Unprompted.shortcode_user_vars else 1,  # p.inpaint_full_res_padding
                0,  # p.inpainting_mask_invert
                "",  #p.batch_input_directory
                "",  #p.batch_output_directory
                "",  #p.img2img_batch_inpaint_mask_dir
                "",  # override_settings_texts
                self.Unprompted.shortcode_user_vars["img2img_batch_use_png_info"] if "img2img_batch_use_png_info" in self.Unprompted.shortcode_user_vars else 0,  # img2img_batch_use_png_info
                [],  # img2img_batch_png_info_props,
                "",  # img2img_batch_png_info_dir
                temp_gr_request,
                *self.Unprompted.main_p.script_args)

However, the img2img function that is being called, located in C:\A1111 Web UI Autoinstaller\stable-diffusion-webui\modules\img2img.py, has a different (incompatible) function signature:

def img2img(id_task: str, request: gr.Request, mode: int, prompt: str, negative_prompt: str, prompt_styles, init_img, sketch, init_img_with_mask, inpaint_color_sketch, inpaint_color_sketch_orig, init_img_inpaint, init_mask_inpaint, mask_blur: int, mask_alpha: float, inpainting_fill: int, n_iter: int, batch_size: int, cfg_scale: float, image_cfg_scale: float, denoising_strength: float, selected_scale_tab: int, height: int, width: int, scale_by: float, resize_mode: int, inpaint_full_res: bool, inpaint_full_res_padding: int, inpainting_mask_invert: int, img2img_batch_input_dir: str, img2img_batch_output_dir: str, img2img_batch_inpaint_mask_dir: str, override_settings_texts, img2img_batch_use_png_info: bool, img2img_batch_png_info_props: list, img2img_batch_png_info_dir: str, *args):

In particular, in the call to this function, the 'request' parameter is being passed in the second-to-last argument position (between img2img_batch_png_info_dir and *self.Unprompted.main_p.script_args), but the function apparently expects to receive it in the second argument position (between id_task and mode). Also, the arguments 'steps' and 'sampler_mode' being passed in the function call don't seem to have any corresponding function parameters at all, which will result in all other arguments after them being off by two positions.

It appears to me that the function signature of the Automatic1111 img2img function has changed, and the unprompted plugin's call to it therefore needs to be changed to match it.

As an experiment, I tried making this change myself (move the request parameter, and delete the steps and sampler_mode parameters), but that resulted in getting a different error:

2024-04-22 00:28:32,497  (INFO)     [Unprompted.after] Processing After content for batch 0, block 0...██████████████████████████████████████████████████████████| 20/20 [00:01<00:00,  9.68it/s]
2024-04-22 00:28:32,498  (WARNING)  [Unprompted.zoom_enhance] Since you are using the debug flag, Zoom Enhance will save test images to the root of your WebUI folder - don't forget!
2024-04-22 00:28:34,256  (WARNING)  [Unprompted.zoom_enhance] Processing mode: Shortcode-Based (may not be compatible with ControlNet)
*** Error running setup: C:\@\Down\Programs\AI\A1111 Web UI Autoinstaller\stable-diffusion-webui\modules/processing_scripts\seed.py
    Traceback (most recent call last):
      File "C:\@\Down\Programs\AI\A1111 Web UI Autoinstaller\stable-diffusion-webui\modules\scripts.py", line 979, in setup_scrips
        script.setup(p, *script_args)
      File "C:\@\Down\Programs\AI\A1111 Web UI Autoinstaller\stable-diffusion-webui\modules/processing_scripts\seed.py", line 76, in setup
        if seed_checkbox and seed_resize_from_w > 0 and seed_resize_from_h > 0:
    TypeError: '>' not supported between instances of 'str' and 'int'

---
2024-04-22 00:28:34,260  (ERROR)    [Unprompted.img2img] Exception while running the img2img task
Traceback (most recent call last):
  File "C:\@\Down\Programs\AI\A1111 Web UI Autoinstaller\stable-diffusion-webui\extensions\unprompted/shortcodes\stable_diffusion\img2img.py", line 110, in run_atomic
    img2img_result = img2img_func(
  File "C:\@\Down\Programs\AI\A1111 Web UI Autoinstaller\stable-diffusion-webui\modules\img2img.py", line 232, in img2img
    processed = process_images(p)
  File "C:\@\Down\Programs\AI\A1111 Web UI Autoinstaller\stable-diffusion-webui\modules\processing.py", line 845, in process_images
    res = process_images_inner(p)
  File "C:\@\Down\Programs\AI\A1111 Web UI Autoinstaller\stable-diffusion-webui\modules\processing.py", line 915, in process_images_inner
    p.init(p.all_prompts, p.all_seeds, p.all_subseeds)
  File "C:\@\Down\Programs\AI\A1111 Web UI Autoinstaller\stable-diffusion-webui\modules\processing.py", line 1582, in init
    self.sampler = sd_samplers.create_sampler(self.sampler_name, self.sd_model)
  File "C:\@\Down\Programs\AI\A1111 Web UI Autoinstaller\stable-diffusion-webui\modules\sd_samplers.py", line 36, in create_sampler
    assert config is not None, f'bad sampler name: {name}'
AssertionError: bad sampler name: -1
2024-04-22 00:28:34,262  (ERROR)    [Unprompted.img2img] The returned object does not appear to contain an image: []
2024-04-22 00:28:34,262  (ERROR)    [Unprompted.zoom_enhance] Exception while running the img2img task
Traceback (most recent call last):
  File "C:\@\Down\Programs\AI\A1111 Web UI Autoinstaller\stable-diffusion-webui\extensions\unprompted/shortcodes\stable_diffusion\zoom_enhance.py", line 398, in run_atomic
    if debug: fixed_image.save(f"zoom_enhance_img{image_idx}_5_enhanced.png")
AttributeError: 'str' object has no attribute 'save'

so there may still be other problems to be fixed.

Also, please notice the "AttributeError: 'str' object has no attribute 'save'" error message that eventually occurs because of these earlier errors. It looks as though an incorrectly typed value is being returned from a function (e.g. returning an error message instead of an image object that was expected by the caller). This situation should probably have been detected in a different way (throw an exception instead of returning a string, or have the caller detect the returned string and handle it as an error?) and should then have resulted in an error message that described what went wrong more clearly, like "Attempt to execute img2img failed due to unexpected exception: <exception error message text goes here>".

When fixing this bug, I suggest looking at other calls to Automatic1111 functions that Unprompted makes (txt2img, etc.), and ensuring that they don't have similar problems. Whoever changed the signature of the img2img function might have changed others as well.

Prompt

See above.

Log output

See above.

Unprompted version

8cc97ad8 | 2024-04-19 00:20:07

WebUI version

v1.9.0 adadb4e3c7382bf3e4f7519126cd6c70f4f8557b, Python 3.10.11

ThereforeGames commented 6 months ago

Hi @Sporking,

Thank you for reaching out and for the detailed bug report.

I was looking into the [img2img] shortcode last night and drew the same conclusions as you have - the underlying WebUI function takes a different set of arguments in v1.9.0. It isn't clear to me yet how to pass in the sampler_name and scheduler values. If anyone knows more, please feel free to chime in.

I'll attach the current, work-in-progress update for [img2img] if anyone wants to take a swing at it. Otherwise, I should have some time later this week to debug.

img2img.zip

ThereforeGames commented 6 months ago

Hi,

After some trial and error, I found I was able to resolve the issue by creating a patched version of the WebUI's img2img() method and supplying it with the missing variables. This approach should be a bit more resilient to future WebUI updates.

I'll attach the fixed shortcode here. It will be included in the Unprompted v10.10.0 release, coming soon™.

img2img_fix_v2.zip

ThereforeGames commented 6 months ago

This fix is now live in v10.10.0. Let me know if it gives you any further trouble. Thank you!