search

Acly / krita-ai-diffusion

Streamlined interface for generating images with AI in Krita. Inpaint and outpaint with optional text prompt, no tweaking required.
https://www.interstice.cloud
GNU General Public License v3.0
6.41k stars 310 forks source link

Selecting an area for live generation no longer works #539

Closed drphero closed 5 months ago

drphero commented 5 months ago

A few months ago, I was using this video to fix hands. https://www.youtube.com/watch?v=sCtRgc21CjA

It worked perfectly before, but now this seems to no longer work after updating everything to 1.16.0. I am still able to use the live generation when starting on a blank canvas however.

When trying to use it on a section of the image like in the video, it makes no changes, even with the strength at 100%.

ComfyUI output:

got prompt
100%|████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 10.39it/s]
Prompt executed in 0.64 seconds

Client log:

2024-03-28 20:23:16,167 INFO Extension initialized, Version: 1.16.0, Python: 3.10.7 (tags/v3.10.7:6cc6b13, Sep  5 2022, 14:08:36) [MSC v.1933 64 bit (AMD64)]
2024-03-28 20:23:16,167 INFO Loading settings from C:\Users\Shadow\AppData\Roaming\krita\ai_diffusion\settings.json
2024-03-28 20:23:17,989 INFO Connecting to http://127.0.0.1:8188
2024-03-28 20:23:18,756 INFO Found ControlNet model inpaint for SD 1.5: control_v11p_sd15_inpaint.pth
2024-03-28 20:23:18,757 INFO Optional ControlNet model scribble for SD 1.5 not found (search path: control_v11p_sd15_scribble, control_lora_rank128_v11p_sd15_scribble)
2024-03-28 20:23:18,757 INFO Found ControlNet model scribble for SD XL: control-lora-sketch-rank128-metadata.safetensors
2024-03-28 20:23:18,758 INFO Found ControlNet model line_art for SD 1.5: control_v11p_sd15_lineart.pth
2024-03-28 20:23:18,758 INFO Found ControlNet model line_art for SD XL: control-lora-sketch-rank128-metadata.safetensors
2024-03-28 20:23:18,759 INFO Found ControlNet model soft_edge for SD 1.5: control_v11p_sd15_softedge.pth
2024-03-28 20:23:18,759 INFO Found ControlNet model canny_edge for SD 1.5: control_v11p_sd15_canny.pth
2024-03-28 20:23:18,759 INFO Found ControlNet model canny_edge for SD XL: control-lora-canny-rank256.safetensors
2024-03-28 20:23:18,760 INFO Found ControlNet model depth for SD 1.5: control_v11f1p_sd15_depth.pth
2024-03-28 20:23:18,760 INFO Found ControlNet model depth for SD XL: control-lora-depth-rank128.safetensors
2024-03-28 20:23:18,760 INFO Optional ControlNet model normal for SD 1.5 not found (search path: control_v11p_sd15_normalbae, control_lora_rank128_v11p_sd15_normalbae)
2024-03-28 20:23:18,761 INFO Found ControlNet model pose for SD 1.5: control_v11p_sd15_openpose.pth
2024-03-28 20:23:18,761 INFO Found ControlNet model pose for SD XL: thibaud_xl_openpose.safetensors
2024-03-28 20:23:18,761 INFO Found ControlNet model segmentation for SD 1.5: control_v11p_sd15_seg.pth
2024-03-28 20:23:18,762 INFO Found ControlNet model blur for SD 1.5: control_v11f1e_sd15_tile.pth
2024-03-28 20:23:18,762 INFO Optional ControlNet model blur for SD XL not found (search path: ttplanetsdxlcontrolnet, ttplanet_sdxl_controlnet_tile_realistic)
2024-03-28 20:23:18,763 INFO Optional ControlNet model stencil for SD 1.5 not found (search path: control_v1p_sd15_qrcode_monster)
2024-03-28 20:23:18,763 INFO Found ControlNet model hands for SD 1.5: control_sd15_inpaint_depth_hand_fp16.safetensors
2024-03-28 20:23:18,763 INFO Found ControlNet model hands for SD XL: control-lora-depth-rank128.safetensors
2024-03-28 20:23:18,764 INFO Found CLIP Vision model ip_adapter for All: SD1.5\model.safetensors
2024-03-28 20:23:18,764 INFO Found IP-Adapter model reference for SD 1.5: ip-adapter_sd15.safetensors
2024-03-28 20:23:18,764 WARNING Missing IP-Adapter model reference for SD XL
2024-03-28 20:23:18,765 INFO -> No model matches search paths: ip-adapter_sdxl_vit-h
2024-03-28 20:23:18,765 INFO -> Available models: ip-adapter-faceid-plusv2_sd15.bin, ip-adapter-faceid-portrait_sd15.bin, ip-adapter-plus-face_sd15.bin, ip-adapter-plus_sd15.safetensors, ip-adapter.bin, ip-adapter_sd15.safetensors
2024-03-28 20:23:18,765 INFO Found IP-Adapter model face for SD 1.5: ip-adapter-faceid-plusv2_sd15.bin
2024-03-28 20:23:18,766 INFO Optional IP-Adapter model face for SD XL not found (search path: ip-adapter-faceid-plusv2_sdxl, ip-adapter-faceid_sdxl)
2024-03-28 20:23:18,766 INFO Found Upscale model default for All: 4x_NMKD-Superscale-SP_178000_G.pth
2024-03-28 20:23:18,766 INFO Found Upscale model fast_2x for All: OmniSR_X2_DIV2K.safetensors
2024-03-28 20:23:18,767 INFO Found Upscale model fast_3x for All: OmniSR_X3_DIV2K.safetensors
2024-03-28 20:23:18,767 INFO Found Upscale model fast_4x for All: OmniSR_X4_DIV2K.safetensors
2024-03-28 20:23:18,767 INFO Found Inpaint model default for All: MAT_Places512_G_fp16.safetensors
2024-03-28 20:23:18,768 INFO Found Inpaint model fooocus_head for SD XL: fooocus_inpaint_head.pth
2024-03-28 20:23:18,768 INFO Found Inpaint model fooocus_patch for SD XL: inpaint_v26.fooocus.patch
2024-03-28 20:23:18,768 INFO Found LoRA model lcm for SD 1.5: lcm-lora-sdv1-5.safetensors
2024-03-28 20:23:18,769 INFO Found LoRA model lcm for SD XL: lcm-lora-sdxl.safetensors
2024-03-28 20:23:18,770 INFO Found LoRA model face for SD 1.5: ip-adapter-faceid-plusv2_sd15_lora.safetensors
2024-03-28 20:23:18,770 INFO Found LoRA model face for SD XL: ip-adapter-faceid-plusv2_sdxl_lora.safetensors
2024-03-28 20:23:21,293 INFO SD 1.5: supported
2024-03-28 20:23:21,293 INFO SD XL: missing 1 models
drphero commented 5 months ago

Forgot to add that the regular generation mode does still work when only a part of the image is selected.

agolinux3 commented 5 months ago

Same here. For example, I tried to change a color in some selection of my image but now it does nothing. Work fine with 1.15.

figihiuy commented 5 months ago

I get the same , seems to be a bug. It seems to work in Generate mode.

Acly commented 5 months ago

It seems to work for me.

  1. I open an image
  2. Start live mode
  3. Draw a selection

-> get preview of selected area with generated content

Anything else you're doing or something different? Does it show preview of the selected area, but it just doesn't change anything? Is it exactly the same when you apply?

Also if someone can share workflow it might give a hint (enable Interface -> Dump Workflow in settings, find the workflow.json file in the logs folder)

vidiotgameboss commented 5 months ago

Same issue here, selecting an area works but it only seems to generate near the bottom edge of the selection rather than generating in the entire selection.

Maxxing out the padding seems to fix it but only sometimes, not always. Can confirm this started happening since the cloud branch changes, more specifically it's probably related to the differential diffusion commit.

DonDawsy commented 5 months ago

I have the same issue, it seems like there's something wrong with the mask generation on Live compared to Generate.

I converted the base64 from the masks in the workflows on inpainting with Generate and Live and got these results:

Generate (Correct resolution) Generate1

Live: Live1

Acly commented 5 months ago

Got it, thanks. Staying within the "native checkpoint resolution" (512~768 for SD1.5) works fine, bigger images show some mask mix up...

Acly commented 5 months ago

Fixed in v1.16.1