[Bug]: ADetailer breaks Regional Prompter after the first batch

Turamarth14 commented 1 month ago

Describe the bug

When creating images using the Regional Prompter extension, repeating the creation with the same settings and an activated ADetailer completely changes the images after the first batch. It seems that the prompt for the last region gets lost and instead the prompt of the previous region is used. The prompt information stored within the image is also changed, which might be related. For the first image the prompt information is the same with and without ADetailer. For the second image it changes. Without Adetailer:

parameters score_9, score_6_up, source_anime, 2girls, leotard, flexing biceps, gym,

BREAK brown hair, green eyes, ponytail, black leotard,
BREAK blonde hair, blue eyes, short hair, white leotard
Negative prompt: 3d, monochrome, simple background

With Adetailer:

parameters score_9, score_6_up, source_anime, 2girls, leotard, flexing biceps, gym,

,  brown hair, green eyes, ponytail, black leotard,
BREAK score_9, score_6_up, source_anime, 2girls, leotard, flexing biceps, gym,

,  blonde hair, blue eyes, short hair, white leotard
Negative prompt: 3d, monochrome, simple background

Steps to reproduce

Create two batches of a single image with multiple regions using Regional Prompter and deactivated ADetailer. In Regional Prompter activate Use common prompt and set the splitting to columns with a divide ratio of 1,1. For example:
```
score_9, score_6_up, source_anime, 2girls, leotard, flexing biceps, gym,
BREAK brown hair, green eyes, ponytail, black leotard,
BREAK blonde hair, blue eyes, short hair, white leotard
```
Activate ADetailer and repeat the image generation without changing any other settings.

Screenshots

Image generation settings: GenerationSettings

Regional Prompter settings: RegionalPrompterSettings

ADetailer settings: AdetailerSettings

Images without activated ADetailer: ImagesWithoutAdetailer

Images with activated ADetailer: ImagesWithAdetailer

Console logs, from start to end.

C:\Users\admin\stable-diffusion\stable-diffusion-webui>webui-user.bat
venv "C:\Users\admin\stable-diffusion\stable-diffusion-webui\venv\Scripts\Python.exe"
initial startup: done in 0.950s
  prepare environment:
  checks: done in 0.012s
  git version info: done in 29.341s
Python 3.10.8 (tags/v3.10.8:aaaf517, Oct 11 2022, 16:50:30) [MSC v.1933 64 bit (AMD64)]
Version: v1.9.4-253-gb2453d28
Commit hash: b2453d280acd5d58e6f4f0e92b7ea5011b0a6fa4
  torch GPU test: done in 46.336s
  clone repositores: done in 1.073s
    run extensions installers:
    a1111-sd-webui-tagcomplete: done in 0.010s
    adetailer: done in 0.961s
    IF_prompt_MKR: done in 0.205s
    multidiffusion-upscaler-for-automatic1111: done in 0.000s
    sd-extension-steps-animation: done in 0.001s
    sd-extension-system-info: done in 0.000s
    SD-WebUI-BatchCheckpointPrompt: done in 0.001s
Installing sd-webui-controlnet requirement: changing controlnet_aux version from None to 0.0.9
    sd-webui-controlnet: done in 74.001s
    sd-webui-facefusion: done in 6.351s
    sd-webui-infinite-image-browsing: done in 0.813s
    sd-webui-negpip: done in 0.033s
CUDA 12.1
    sd-webui-reactor: done in 2.908s
    sd-webui-regional-prompter: done in 0.000s
    sdweb-clip-changer: done in 0.000s
    sd_webui_SAG: done in 0.000s
    shift-attention: done in 0.294s
    Stable-Diffusion-Webui-Civitai-Helper: done in 0.070s
    stable-diffusion-webui-two-shot: done in 0.034s
    z-a1111-sd-webui-dtg: done in 0.658s
...
  update extensions: done in 176.639s
Launching Web UI with arguments: --medvram --no-half-vae --xformers --log-startup --update-all-extensions
launcher: done in 2.168s
import torch: done in 4.572s
import torch: done in 224.878s
import gradio: done in 45.743s
setup paths: done in 25.657s
import ldm: done in 0.546s
import sgm: done in 0.001s
initialize shared: done in 8.957s
other imports: done in 53.639s
opts onchange: done in 0.001s
setup SD model: done in 0.035s
setup codeformer: done in 0.141s
setup gfpgan: done in 4.123s
set samplers: done in 0.000s
list extensions: done in 0.005s
restore config state file: done in 0.000s
list SD models: done in 27.521s
list localizations: done in 0.051s
  load scripts:
  GIF_Slice.py: done in 1.708s
  GIF_Slice_Enhanced.py: done in 0.086s
  custom_code.py: done in 0.111s
  img2imgalt.py: done in 0.077s
  loopback.py: done in 0.053s
  outpainting_mk_2.py: done in 0.014s
  poor_mans_outpainting.py: done in 0.022s
  postprocessing_codeformer.py: done in 0.044s
  postprocessing_gfpgan.py: done in 0.040s
  postprocessing_upscale.py: done in 0.047s
  prompt_matrix.py: done in 0.041s
  prompts_from_file.py: done in 0.017s
  sd_upscale.py: done in 0.001s
  xyz_grid.py: done in 0.026s
  ldsr_model.py: done in 1.691s
  lora_script.py: done in 24.962s
  scunet_model.py: done in 0.520s
  swinir_model.py: done in 0.261s
  hotkey_config.py: done in 0.068s
  extra_options_section.py: done in 0.142s
  hypertile_script.py: done in 0.193s
  postprocessing_autosized_crop.py: done in 0.089s
  postprocessing_caption.py: done in 0.078s
  postprocessing_create_flipped_copies.py: done in 0.002s
  postprocessing_focal_crop.py: done in 0.092s
  postprocessing_split_oversized.py: done in 0.002s
  soft_inpainting.py: done in 0.061s
  extrafiles.py: done in 0.029s
  if_prompt_mkr.py: done in 12.799s
  modeltags.py: done in 0.001s
  BatchParams.py: done in 0.641s
  CivitaihelperPrompts.py: done in 0.135s
  Logger.py: done in 0.003s
  Save.py: done in 0.137s
  Utils.py: done in 0.003s
  batchCheckpointsPrompt.py: done in 0.086s
  settings.py: done in 0.432s
CHv1.8.10: Get Custom Model Folder
  civitai_helper.py: done in 4.406s
  model_keyword_support.py: done in 0.166s
  shared_paths.py: done in 0.002s
  tag_autocomplete_helper.py: done in 4.417s
  tag_frequency_db.py: done in 0.001s
[-] ADetailer initialized. version: 24.6.0, num models: 10
  !adetailer.py: done in 2.538s
  tilediffusion.py: done in 0.423s
  tileglobal.py: done in 0.263s
  tilevae.py: done in 0.213s
  steps_animation.py: done in 0.057s
  system-info.py: done in 0.213s
  adapter.py: done in 0.007s
ControlNet preprocessor location: C:\Users\admin\stable-diffusion\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads
  api.py: done in 4.441s
  batch_hijack.py: done in 0.051s
  cldm.py: done in 0.025s
2024-07-17 09:27:25,322 - ControlNet - INFO - ControlNet v1.1.455
  controlnet.py: done in 18.405s
  controlnet_diffusers.py: done in 0.001s
  controlnet_lllite.py: done in 0.000s
  controlnet_lora.py: done in 0.001s
  controlnet_model_guess.py: done in 0.001s
  controlnet_sparsectrl.py: done in 0.000s
  controlnet_version.py: done in 0.000s
  enums.py: done in 0.002s
  external_code.py: done in 0.000s
  global_state.py: done in 0.001s
  hook.py: done in 0.001s
  infotext.py: done in 0.000s
  logging.py: done in 0.001s
  lvminthin.py: done in 0.000s
  movie2movie.py: done in 0.034s
  supported_preprocessor.py: done in 0.009s
  utils.py: done in 0.003s
  xyz_grid_support.py: done in 0.020s
Using TensorFlow backend
  facefusion_ui.py: done in 25.619s
  iib_setup.py: done in 0.810s
  negpip.py: done in 0.192s
  console_log_patch.py: done in 0.189s
  reactor_api.py: done in 0.489s
09:27:53 - ReActor - STATUS - Running v0.7.1-a1 on Device: CUDA
  reactor_faceswap.py: done in 0.290s
  reactor_globals.py: done in 0.005s
  reactor_helpers.py: done in 0.000s
  reactor_logger.py: done in 0.001s
  reactor_swapper.py: done in 0.002s
  reactor_version.py: done in 0.001s
  reactor_xyz.py: done in 0.069s
  attention.py: done in 0.023s
  latent.py: done in 0.106s
  regions.py: done in 0.003s
  rp.py: done in 0.220s
  rps.py: done in 0.086s
  SAG.py: done in 0.117s
  xyz_grid_support_sag.py: done in 0.002s
  sdweb_clip_changer.py: done in 0.226s
  shift_attention.py: done in 0.400s
  shift_attention_rife_fixed.py: done in 0.021s
  sketch_helper.py: done in 0.033s
  two_shot.py: done in 0.119s
  dtg.py: done in 2.026s
  comments.py: done in 0.142s
  refiner.py: done in 0.027s
  sampler.py: done in 0.055s
  seed.py: done in 0.040s
load upscalers: done in 0.403s
refresh VAE: done in 0.100s
refresh textual inversion templates: done in 0.154s
scripts list_optimizers: done in 0.008s
scripts list_unets: done in 0.000s
reload hypernetworks: done in 0.008s
Loading weights [d3ee23d452] from C:\Users\admin\stable-diffusion\stable-diffusion-webui\models\Stable-diffusion\autismmixSDXL_autismmixDPO.safetensors
initialize extra networks: done in 2.761s
scripts before_ui_callback: done in 0.356s
CHv1.8.10: Set Proxy:
Character Path: C:\Users\admin\text-generation-webui\characters
Character List: ['iF_Ai_SD', 'iF_Ai_SD_b']
Embedding Model value: None
LORA Model value: None
2024-07-17 09:28:16,651 - ControlNet - INFO - ControlNet UI callback registered.
Character Path: C:\Users\admin\text-generation-webui\characters
Character List: ['iF_Ai_SD', 'iF_Ai_SD_b']
Embedding Model value: None
LORA Model value: None
Creating model from config: C:\Users\admin\stable-diffusion\stable-diffusion-webui\repositories\generative-models\configs\inference\sd_xl_base.yaml
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: C:\Users\admin/.insightface\models\buffalo_l\1k3d68.onnx landmark_3d_68 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: C:\Users\admin/.insightface\models\buffalo_l\2d106det.onnx landmark_2d_106 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: C:\Users\admin/.insightface\models\buffalo_l\det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: C:\Users\admin/.insightface\models\buffalo_l\genderage.onnx genderage ['None', 3, 96, 96] 0.0 1.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: C:\Users\admin/.insightface\models\buffalo_l\w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5
set det-size: (640, 640)
create ui: done in 29.876s
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
gradio launch: done in 6.704s
add APIs: done in 0.089s
  app_started_callback:
  lora_script.py: done in 0.001s
  tag_autocomplete_helper.py: done in 0.006s
  !adetailer.py: done in 0.000s
  system-info.py: done in 0.004s
  api.py: done in 0.004s
IIB Database file has been successfully backed up to the backup folder.
  iib_setup.py: done in 8.051s
  reactor_api.py: done in 0.002s
found lora_script.py. exchange my callback index 2 -> 0
CLIP Changer.on_app_started done.
  sdweb_clip_changer.py: done in 0.001s
Startup time: 899.0s (initial startup: 1.0s, prepare environment: 343.7s, launcher: 2.2s, import torch: 229.5s, import gradio: 45.7s, setup paths: 25.7s, import ldm: 0.5s, initialize shared: 9.0s, other imports: 53.6s, setup codeformer: 0.1s, setup gfpgan: 4.1s, list SD models: 27.5s, load scripts: 111.7s, load upscalers: 0.4s, refresh textual inversion templates: 0.2s, initialize extra networks: 2.8s, scripts before_ui_callback: 0.4s, create ui: 29.9s, gradio launch: 6.7s, app_started_callback: 8.1s).
Loading VAE weights specified in settings: C:\Users\admin\stable-diffusion\stable-diffusion-webui\models\Stable-diffusion\sdxl.vae.safetensors
Applying attention optimization: xformers... done.
Model loaded in 333.6s (load weights from disk: 21.6s, load config: 0.2s, create model: 2.5s, apply weights to model: 290.7s, load VAE: 7.5s, hijack: 0.2s, load textual inversion embeddings: 5.0s, calculate empty prompt: 5.8s).
1,1 0.2 Horizontal
Regional Prompter Active, Pos tokens : [40, 40], Neg tokens : [7]
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:26<00:00,  1.13it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:24<00:00,  1.22it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 60/60 [04:55<00:00,  4.93s/it]
1,1 0.2 Horizontal0%|██████████████████████████████████████████████████████████████████| 60/60 [04:55<00:00,  1.30it/s]
Regional Prompter Active, Pos tokens : [40, 40], Neg tokens : [7]
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:25<00:00,  1.17it/s]
Total progress:  50%|█████████████████████████████████                                 | 30/60 [00:22<00:24,  1.24it/s]
0: 448x640 2 faces, 1962.2ms
Speed: 375.7ms preprocess, 1962.2ms inference, 250.3ms postprocess per image at shape (1, 3, 448, 640)
WARNING:root:Sampler Scheduler autocorrection: "Euler a" -> "Euler a", "None" -> "Automatic"
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:24<00:00,  1.25it/s]
WARNING:root:Sampler Scheduler autocorrection: "Euler a" -> "Euler a", "None" -> "Automatic"
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:23<00:00,  1.27it/s]
1,1 0.2 Horizontal
Regional Prompter Active, Pos tokens : [40, 40], Neg tokens : [7]
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:24<00:00,  1.24it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 60/60 [02:16<00:00,  1.30it/s]
0: 448x640 2 faces, 57.3ms
Speed: 3.0ms preprocess, 57.3ms inference, 1.0ms postprocess per image at shape (1, 3, 448, 640)
WARNING:root:Sampler Scheduler autocorrection: "Euler a" -> "Euler a", "None" -> "Automatic"
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:23<00:00,  1.29it/s]
WARNING:root:Sampler Scheduler autocorrection: "Euler a" -> "Euler a", "None" -> "Automatic"
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:23<00:00,  1.27it/s]
1,1 0.2 Horizontal
Regional Prompter Active, Pos tokens : [40, 40], Neg tokens : [7]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 60/60 [03:46<00:00,  3.77s/it]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 60/60 [03:46<00:00,  1.30it/s]

List of installed extensions

IF_prompt_MKR SD-WebUI-BatchCheckpointPrompt Stable-Diffusion-Webui-Civitai-Helper a1111-sd-webui-tagcomplete adetailer multidiffusion-upscaler-for-automatic1111 sd-extension-steps-animation sd-extension-system-info sd-webui-controlnet sd-webui-facefusion sd-webui-infinite-image-browsing sd-webui-negpip sd-webui-reactor sd-webui-regional-prompter sd_webui_SAG sdweb-clip-changer shift-attention stable-diffusion-webui-two-shot z-a1111-sd-webui-dtg

dosubot[bot] commented 1 month ago

Hello @Turamarth14! I'm here to assist you with any bugs, questions, or contributions.

To resolve the issue where repeating image creation with the same settings and an activated ADetailer in the Regional Prompter extension changes the images after the first batch, seemingly losing the prompt for the last region, you need to ensure that the prompt is correctly set for each iteration. The i2i_prompts_replace method is responsible for setting the prompt and negative prompt for each iteration.

Here is the relevant code snippet that shows how the prompt is set:

@staticmethod
def i2i_prompts_replace(
    i2i, prompts: list[str], negative_prompts: list[str], j: int
) -> None:
    i1 = min(j, len(prompts) - 1)
    i2 = min(j, len(negative_prompts) - 1)
    prompt = prompts[i1]
    negative_prompt = negative_prompts[i2]
    i2i.prompt = prompt
    i2i.negative_prompt = negative_prompt

Ensure that this method is called correctly for each batch to set the appropriate prompt and negative prompt. If the prompt is not being set correctly, it could lead to the issue you are experiencing ^[1].

To continue talking to Dosu, mention @dosu.

Bing-su commented 1 month ago

Thank you for your report. I've worked on this before with the dynamic prompt extension, and it looks like this issue is related to that. But with issues like this with other extensions, it's hard to know exactly what's causing it, and hard to think of a way to fix it, sorry.

vaskovet2 commented 4 weeks ago

Try using the Token "[SEP]" in the a detailer to separated the area into two

Turamarth14 commented 4 weeks ago

Try using the Token "[SEP]" in the a detailer to separated the area into two

That makes no difference. The second generated image is still completely different from the one without active ADetailer. ImagesWithAdetailer_SEP

Bing-su / adetailer