search

Metachs / sdwebui-nai-api

zlib License
22 stars 3 forks source link

Using Adetailer + nai extensions and batch count > 1 gets stuck when giving output images. #4

Closed Panchovix closed 11 months ago

Panchovix commented 11 months ago

Nice work! I'm having an issue with latest updates where when trying to gen an image with both adetailer + nai extension, it gets stuck and never ends.

Using no adetailer works.

Console log when not using adetailer:

Patching Image Processing for NAI Generator Script.
Unpatching Image Processing for NAI Generator Script.
nai_preprocess
nai_generate_images
Loading Images:  0 1 1
Loading Image 0
{"input":"smiling, best quality, amazing quality, very aesthetic, absurdres","model":"nai-diffusion-3","action":"generate","parameters":{"width":832,"height":1216,"scale":4.5,"sampler":"k_euler","steps":28,"seed":3236620642,"n_samples":1,"ucPreset":0,"qualityToggle":"true","sm":"false","sm_dyn":"false","dynamic_thresholding":"false","controlnet_strength":1,"legacy":"false","add_original_image":"True","uncond_scale":1,"cfg_rescale":0,"noise_schedule":"native","negative_prompt":"lowres, {bad}, error, fewer, extra, missing, worst quality, jpeg artifacts, bad quality, watermark, unfinished, displeasing, chromatic aberration, signature, extra digits, artistic error, username, scan, [abstract], "}}
Query Complete 0
Reading Result Images 0
Read Image: 0 0
Number of Images:  1
False False False False

Console log when using adetailer

Patching Image Processing for NAI Generator Script.
Unpatching Image Processing for NAI Generator Script.
nai_preprocess
nai_generate_images
Loading Images:  0 1 1
Loading Image 0
{"input":"smiling, best quality, amazing quality, very aesthetic, absurdres","model":"nai-diffusion-3","action":"generate","parameters":{"width":832,"height":1216,"scale":4.5,"sampler":"k_euler","steps":28,"seed":3236620642,"n_samples":1,"ucPreset":0,"qualityToggle":"true","sm":"false","sm_dyn":"false","dynamic_thresholding":"false","controlnet_strength":1,"legacy":"false","add_original_image":"True","uncond_scale":1,"cfg_rescale":0,"noise_schedule":"native","negative_prompt":"lowres, {bad}, error, fewer, extra, missing, worst quality, jpeg artifacts, bad quality, watermark, unfinished, displeasing, chromatic aberration, signature, extra digits, artistic error, username, scan, [abstract], "}}
Query Complete 0
Reading Result Images 0
Read Image: 0 0
Number of Images:  1
False False False False

0: 640x448 1 face, 65.5ms
Speed: 3.0ms preprocess, 65.5ms inference, 29.0ms postprocess per image at shape (1, 3, 640, 448)
Patching Image Processing for NAI Generator Script.
Unpatching Image Processing for NAI Generator Script.
nai_preprocess
init_images:  1
832 1216
832 1216 832 1216 832 1216 832 1216
nai_generate_images
Loading Images:  0 1 1
Loading Image 0
{"input":"smiling, best quality, amazing quality, very aesthetic, absurdres","model":"nai-diffusion-3-inpainting","action":"infill","parameters":{"width":832,"height":1216,"scale":4.5,"sampler":"k_euler","steps":28,"seed":3236620642,"n_samples":1,"strength":0.4,"noise":0,"ucPreset":0,"qualityToggle":"true","sm":"false","sm_dyn":"false","dynamic_thresholding":"false","controlnet_strength":1,"legacy":"false","add_original_image":"False","uncond_scale":1,"cfg_rescale":0,"noise_schedule":"native","image":"","mask":"","extra_noise_seed":3236620642,"negative_prompt":"lowres, {bad}, error, fewer, extra, missing, worst quality, jpeg artifacts, bad quality, watermark, unfinished, displeasing, chromatic aberration, signature, extra digits, artistic error, username, scan, [abstract], "}}
Query Complete 0
Reading Result Images 0
Read Image: 0 0
Number of Images:  1

And then it gets stuck. Only way to make it work again is restarting webui.

Metachs commented 11 months ago

Great, that at least pins it down to a single function, just pushed an experimental change that might resolve the issue.

Panchovix commented 11 months ago

@Metachs just was testing it and got another issue.

Traceback (most recent call last):
      File "G:\Stable difussion\stable-diffusion-webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
                   ^^^^^^^^^^^^^^^^^^^^^
      File "G:\Stable difussion\stable-diffusion-webui\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^
      File "G:\Stable difussion\stable-diffusion-webui\modules\txt2img.py", line 55, in txt2img
        processed = processing.process_images(p)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "G:\Stable difussion\stable-diffusion-webui\modules\processing.py", line 734, in process_images
        res = process_images_inner(p)
              ^^^^^^^^^^^^^^^^^^^^^^^
      File "G:\Stable difussion\stable-diffusion-webui\extensions\sdwebui-nai-api\nai_api_gen\nai_api_processing.py", line 106, in process_images_patched
        results = modules.processing.process_images_pre_patch_4_nai(p)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "G:\Stable difussion\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 42, in processing_process_images_hijack
        return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "G:\Stable difussion\stable-diffusion-webui\modules\processing.py", line 798, in process_images_inner
        p.scripts.process(p)
      File "G:\Stable difussion\stable-diffusion-webui\extensions\sdwebui-nai-api\nai_api_gen\nai_api_processing.py", line 98, in process_patched
        script.nai_image_processsing(p, *args) # Sets p.nai_processed if images have been processed
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "G:\Stable difussion\stable-diffusion-webui\extensions\sdwebui-nai-api\nai_api_gen\nai_api_script.py", line 386, in nai_image_processsing
        if not self.use_batch_processing: self.nai_generate_images(p, *args, **kwargs)
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "G:\Stable difussion\stable-diffusion-webui\extensions\sdwebui-nai-api\nai_api_gen\nai_api_script.py", line 459, in nai_generate_images
        self.get_batch_images(p, getparams, save_images = isimg2img and getattr(p,"inpaint_full_res",False) and shared.opts.data.get('nai_api_save_fragments', False), save_suffix ="-nai-init-image" if do_local_img2img > 0 else "" ,dohash = False, query_batch_size=self.query_batch_size, overlay = inpaint_mode == 1 or getattr(p,"inpaint_full_res",False))
      File "G:\Stable difussion\stable-diffusion-webui\extensions\sdwebui-nai-api\nai_api_gen\nai_api_script.py", line 527, in get_batch_images
        image,code =  nai_api.LOAD(result, parameters)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "G:\Stable difussion\stable-diffusion-webui\extensions\sdwebui-nai-api\nai_api_gen\nai_api.py", line 56, in LOAD
        if isinstance(requests.exceptions.RequestException):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    TypeError: isinstance expected 2 arguments, got 1

---
Metachs commented 11 months ago

Thanks, accidentally pushed an old file, the second should be fixed now.

Panchovix commented 11 months ago

Thanks. Testing again but the issue seems to persist sadly

Console log is

Patching Image Processing for NAI Generator Script.
Unpatching Image Processing for NAI Generator Script.
nai_preprocess
nai_generate_images
Loading Images:  0 1 1
Loading Image 0
{"input":"smiling, best quality, amazing quality, very aesthetic, absurdres","model":"nai-diffusion-3","action":"generate","parameters":{"width":832,"height":1216,"scale":4.5,"sampler":"k_euler","steps":25,"seed":1858508310,"n_samples":1,"ucPreset":0,"qualityToggle":"true","sm":"false","sm_dyn":"false","dynamic_thresholding":"false","controlnet_strength":1,"legacy":"false","add_original_image":"True","uncond_scale":1,"cfg_rescale":0,"noise_schedule":"native","negative_prompt":"lowres, {bad}, error, fewer, extra, missing, worst quality, jpeg artifacts, bad quality, watermark, unfinished, displeasing, chromatic aberration, signature, extra digits, artistic error, username, scan, [abstract], "}}
Query Complete 0
Reading Result Images 0
Read Image: 0 0
Number of Images:  1
False False False False

0: 640x448 1 face, 70.5ms
Speed: 8.0ms preprocess, 70.5ms inference, 48.0ms postprocess per image at shape (1, 3, 640, 448)
Patching Image Processing for NAI Generator Script.
Unpatching Image Processing for NAI Generator Script.
nai_preprocess
init_images:  1
832 1216
832 1216 832 1216 832 1216 832 1216
nai_generate_images
Loading Images:  0 1 1
Loading Image 0
{"input":"smiling, best quality, amazing quality, very aesthetic, absurdres","model":"nai-diffusion-3-inpainting","action":"infill","parameters":{"width":832,"height":1216,"scale":4.5,"sampler":"k_euler","steps":25,"seed":1858508310,"n_samples":1,"strength":0.4,"noise":0,"ucPreset":0,"qualityToggle":"true","sm":"false","sm_dyn":"false","dynamic_thresholding":"false","controlnet_strength":1,"legacy":"false","add_original_image":"False","uncond_scale":1,"cfg_rescale":0,"noise_schedule":"native","image":"","mask":"","extra_noise_seed":1858508310,"negative_prompt":"lowres, {bad}, error, fewer, extra, missing, worst quality, jpeg artifacts, bad quality, watermark, unfinished, displeasing, chromatic aberration, signature, extra digits, artistic error, username, scan, [abstract], "}}
Query Complete 0
Reading Result Images 0
Read Image: 0 0
Number of Images:  1

It seems it doesn't go the "last step", or something related to that, because I think it is doing the inpainting/adetailer process, but the webui doesn't output the final image/images or the api doesn't answer with the last image.

Metachs commented 11 months ago

No idea what could be causing this, as a desperate measure, I added an option that does nothing but skip the function at the point where things break: "Skip transferring original metadata to post-processed images.".

Try enabling that, if it doesn't work, something is very wrong.

Panchovix commented 11 months ago

@Metachs Okay just tested with that option and that did the trick. Now it works even with batch size > 1.

Metachs commented 11 months ago

Great, two more questions, can you list your extensions, and check if you are able to load geninfo/metadata from images elsewhere in sdwebui? Need to figure out if this is some sort of conflict with another extension, or some deeper problem in the additions I made to read NAI stealth metadata. Thanks.

Panchovix commented 11 months ago

Sure, I did test on both normal and clean install. Confirmed that with the "Skip transferring original metadata to post-processed images." option enabled on both make adetailer work fine.

For the normal instance I use normally, I have all of these

image

For the clean install, I used just:

This extension (sdwebui-nai-api) Adetailer (https://github.com/Bing-su/adetailer) Dynamic prompts (https://github.com/adieyal/sd-dynamic-prompts/)

Moving an image to the PNG info tab show this.

image

Sending the image to img2img seems to automatically enable NAI, but not sure if it should export the same settings used or not.

txt2img tab options

image

img2img tab options (after using send to img2img on txt2img tab)

image

Panchovix commented 11 months ago

Just tested https://github.com/Metachs/sdwebui-nai-api/commit/f667a86e8544502d2b27df6d445bf3dcd6c24faa and it seems to be fixed it, so gonna close the issue. Thanks!

Panchovix commented 11 months ago

Tested more and had to re-open. It seems with batch count 1 it works fine, but with batch size > 1 it gets stuck again.

Console output

Patching Image Processing for NAI Generator Script.
Unpatching Image Processing for NAI Generator Script.
INFO:sd_dynamic_prompts.dynamic_prompting:Prompt matrix will create 2 images in a total of 1 batches.
nai_preprocess
nai_generate_images
Loading Images:  0 1 2
Loading Image 0
{"input":"smiling, best quality, amazing quality, very aesthetic, absurdres","model":"nai-diffusion-3","action":"generate","parameters":{"width":832,"height":1216,"scale":4.5,"sampler":"k_euler","steps":25,"seed":562174990,"n_samples":1,"ucPreset":0,"qualityToggle":"true","sm":"false","sm_dyn":"false","dynamic_thresholding":"false","controlnet_strength":1,"legacy":"false","add_original_image":"True","uncond_scale":1,"cfg_rescale":0,"noise_schedule":"native","negative_prompt":"lowres, {bad}, error, fewer, extra, missing, worst quality, jpeg artifacts, bad quality, watermark, unfinished, displeasing, chromatic aberration, signature, extra digits, artistic error, username, scan, [abstract], "}}
Query Complete 0
Reading Result Images 0
Read Image: 0 0
Loading Image 1
{"input":"smiling, best quality, amazing quality, very aesthetic, absurdres","model":"nai-diffusion-3","action":"generate","parameters":{"width":832,"height":1216,"scale":4.5,"sampler":"k_euler","steps":25,"seed":562174991,"n_samples":1,"ucPreset":0,"qualityToggle":"true","sm":"false","sm_dyn":"false","dynamic_thresholding":"false","controlnet_strength":1,"legacy":"false","add_original_image":"True","uncond_scale":1,"cfg_rescale":0,"noise_schedule":"native","negative_prompt":"lowres, {bad}, error, fewer, extra, missing, worst quality, jpeg artifacts, bad quality, watermark, unfinished, displeasing, chromatic aberration, signature, extra digits, artistic error, username, scan, [abstract], "}}
Query Complete 1
Reading Result Images 1
Read Image: 0 1
Number of Images:  2
False False False False

0: 640x448 1 face, 60.6ms
Speed: 2.0ms preprocess, 60.6ms inference, 2.0ms postprocess per image at shape (1, 3, 640, 448)
Patching Image Processing for NAI Generator Script.
Unpatching Image Processing for NAI Generator Script.
nai_preprocess
init_images:  1
832 1216
832 1216 832 1216 832 1216 832 1216
nai_generate_images
Loading Images:  0 1 1
Loading Image 0
{"input":"smiling, best quality, amazing quality, very aesthetic, absurdres","model":"nai-diffusion-3-inpainting","action":"infill","parameters":{"width":832,"height":1216,"scale":4.5,"sampler":"k_euler","steps":25,"seed":562174990,"n_samples":1,"strength":0.4,"noise":0,"ucPreset":0,"qualityToggle":"true","sm":"false","sm_dyn":"false","dynamic_thresholding":"false","controlnet_strength":1,"legacy":"false","add_original_image":"False","uncond_scale":1,"cfg_rescale":0,"noise_schedule":"native","image":"","mask":"","extra_noise_seed":562174990,"negative_prompt":"lowres, {bad}, error, fewer, extra, missing, worst quality, jpeg artifacts, bad quality, watermark, unfinished, displeasing, chromatic aberration, signature, extra digits, artistic error, username, scan, [abstract], "}}
Query Complete 0
Reading Result Images 0
Read Image: 0 0
Number of Images:  1
False True False True
False False False False

0: 640x448 1 face, 11.5ms
Speed: 3.0ms preprocess, 11.5ms inference, 2.0ms postprocess per image at shape (1, 3, 640, 448)
Patching Image Processing for NAI Generator Script.
Unpatching Image Processing for NAI Generator Script.
nai_preprocess
init_images:  1
832 1216
832 1216 832 1216 832 1216 832 1216
nai_generate_images
Loading Images:  0 1 1
Loading Image 0
{"input":"smiling, best quality, amazing quality, very aesthetic, absurdres","model":"nai-diffusion-3-inpainting","action":"infill","parameters":{"width":832,"height":1216,"scale":4.5,"sampler":"k_euler","steps":25,"seed":562174990,"n_samples":1,"strength":0.4,"noise":0,"ucPreset":0,"qualityToggle":"true","sm":"false","sm_dyn":"false","dynamic_thresholding":"false","controlnet_strength":1,"legacy":"false","add_original_image":"False","uncond_scale":1,"cfg_rescale":0,"noise_schedule":"native","image":"","mask":"","extra_noise_seed":562174990,"negative_prompt":"lowres, {bad}, error, fewer, extra, missing, worst quality, jpeg artifacts, bad quality, watermark, unfinished, displeasing, chromatic aberration, signature, extra digits, artistic error, username, scan, [abstract], "}}
Query Complete 0
Reading Result Images 0
Read Image: 0 0
Number of Images:  1

Then it gets stuck and have to restart the webui. Re-enabling the new option makes it work as normal.

Metachs commented 11 months ago

Thanks, the recent changes should resolve this issue. seems to have been related to either searching for non-existent metadata in the alpha channel, or sharing a single info dictionary between multiple images, both of which have been resolved. The workaround has been removed as the operation it bypasses no longer exists.