adieyal / sd-dynamic-prompts

A custom script for AUTOMATIC1111/stable-diffusion-webui to implement a tiny template language for random prompt generation
MIT License
1.96k stars 253 forks source link

[Bug]: dynamic prompt Extension Processing Styles & Prompts Incorrectly #623

Open chuckkay opened 9 months ago

chuckkay commented 9 months ago

When using the styles & prompts with dynamic prompts and wildcards, the extension seems to repeatedly prepend the prompt information to each successive image in the batch. This leads to nested and incorrect prompt info for each image.

Steps to Reproduce: Create a style for image-to-image batch mode with the following content: photo of {prompt} at the beach Enter "a boy" in the prompt. Add a directory to the img2img batch (e.g., c:\myimages). add some controlnets and adetailer (but still happens even if you dont)

Expected Result: All images in the batch directory should be processed with the prompt info: photo of a boy at the beach

Actual Result: The images are processed as: image 1: photo of a boy at the beach image 2: photo of photo of a boy at the beach at the beach image 3: photo of photo of photo of a boy at the beach at the beach image 4: photo of photo of photo of a boy at the beach at the beach etc.....

Further Observations: When using dynamic prompts and wildcards in the style: photo of {prompt}, {brown_eyes|blue_eyes|green_eyes}, in a { Myfavlocations }

image 1:photo of a woman, brown_eyes, in a scifi movie image 2:photo of photo of a woman, green_eyes, in a action movie, green_eyes, in a space capsule
image 3:photo of photo of photo of a woman, green_eyes, in a action movie, green_eyes, in a horror movie, blue_eyes, in a action movie

results should have been:

image 1:photo of a woman, brown_eyes, in a scifi movie image 2:photo of a woman, green_eyes, in a space capsule image 3:photo of a woman, blue_eyes, in a action movie

log: Already up to date. venv "C:\AI\Automatic-SDXXXL\stable-diffusion-webui\venv\Scripts\Python.exe" Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)] Version: v1.6.0-124-g92464233 Commit hash: 924642331b13b73adbada3270a52cc5c7ae85ace current transparent-background 1.2.4 Installing requirements for face crop img2img Installing opencv-python Installing opencv-python for pixel extension Checking ReActor requirements... Ok Launching Web UI with arguments: No module 'xformers'. Proceeding without it. Tag Autocomplete: Could not locate model-keyword extension, Lora trigger word completion will be limited to those added through the extra networks menu. [-] ADetailer initialized. version: 23.9.1, num models: 9 2023-09-09 18:01:12,092 - ControlNet - INFO - ControlNet v1.1.409 ControlNet preprocessor location: C:\AI\Automatic-SDXXXL\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads 2023-09-09 18:01:12,296 - ControlNet - INFO - ControlNet v1.1.409 18:01:12 - ReActor - STATUS - Running v0.4.1 Loading weights [fa1224c923] from C:\AI\Automatic-SDXXXL\stable-diffusion-webui\models\Stable-diffusion\ICBINP - -I Can't Believe It's Not Photography-\ICBINP - -I Can't Believe It's Not Photography--SECO\icbinpICantBelieveIts_seco.safetensors Creating model from config: C:\AI\Automatic-SDXXXL\stable-diffusion-webui\configs\v1-inference.yaml Civitai Shortcut: v1.6.6 Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch(). Startup time: 24.6s (prepare environment: 11.9s, import torch: 2.7s, import gradio: 1.3s, setup paths: 0.6s, initialize shared: 0.2s, other imports: 1.2s, setup codeformer: 0.1s, list SD models: 0.3s, load scripts: 3.6s, create ui: 2.3s, gradio launch: 0.4s). Applying attention optimization: Doggettx... done. Model loaded in 5.0s (load weights from disk: 1.3s, create model: 0.3s, apply weights to model: 1.3s, load textual inversion embeddings: 1.3s, calculate empty prompt: 0.7s). Will process 494 images, creating 1 new images for each. 2023-09-09 18:21:44,647 - ControlNet - STATUS - Loading preprocessor: revision_clipvision 2023-09-09 18:21:44,647 - ControlNet - STATUS - preprocessor resolution = 768 2023-09-09 18:21:44,661 - ControlNet - STATUS - ControlNet Hooked - Time = 0.02100086212158203 Tile 1/15 Tile 2/15 Tile 3/15 Tile 4/15 Tile 5/15 Tile 6/15 Tile 7/15 Tile 8/15 Tile 9/15 Tile 10/15 Tile 11/15 Tile 12/15 Tile 13/15 Tile 14/15 Tile 15/15 100%|██████████████████████████████████████████████████████████████████████████████████| 11/11 [00:04<00:00, 2.25it/s] mediapipe: 1 detected. | 11/5434 [00:04<39:18, 2.30it/s] 2023-09-09 18:21:51,791 - ControlNet - WARNING - [openpose_full.processor_res] Invalid value(-1), using default value 512. 2023-09-09 18:21:51,791 - ControlNet - STATUS - Loading model from cache: control_v11p_sd15_openpose [cab727d4] 2023-09-09 18:21:51,826 - ControlNet - STATUS - Loading preprocessor: openpose_full 2023-09-09 18:21:51,826 - ControlNet - STATUS - preprocessor resolution = 768 2023-09-09 18:21:53,350 - ControlNet - STATUS - ControlNet Hooked - Time = 1.569061517715454 Tile 1/4 Tile 2/4 Tile 3/4 Tile 4/4 100%|████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:03<00:00, 1.60it/s] [-] ADetailer: applied 1st ad_prompt: 'photo of photo of a woman, green_eyes, in a horror movie, green_eyes, in a action movie' [-] ADetailer: applied 1st ad_negative_prompt: 'FastNegativeV2, ugly, ugly' 2023-09-09 18:21:58,609 - ControlNet - STATUS - Loading preprocessor: revision_clipvision 2023-09-09 18:21:58,609 - ControlNet - STATUS - preprocessor resolution = 768 2023-09-09 18:21:58,628 - ControlNet - STATUS - ControlNet Hooked - Time = 0.2008070945739746 2023-09-09 18:21:58,883 - ControlNet - STATUS - Loading preprocessor: revision_clipvision 2023-09-09 18:21:58,883 - ControlNet - STATUS - preprocessor resolution = 768 2023-09-09 18:21:58,899 - ControlNet - STATUS - ControlNet Hooked - Time = 0.023000240325927734 Tile 1/15 Tile 2/15 Tile 3/15 Tile 4/15 Tile 5/15 Tile 6/15 Tile 7/15 Tile 8/15 Tile 9/15 Tile 10/15 Tile 11/15 Tile 12/15 Tile 13/15 Tile 14/15 Tile 15/15 100%|██████████████████████████████████████████████████████████████████████████████████| 11/11 [00:04<00:00, 2.21it/s] mediapipe: 1 detected. | 22/5434 [00:18<46:55, 1.92it/s] 2023-09-09 18:22:05,817 - ControlNet - WARNING - [openpose_full.processor_res] Invalid value(-1), using default value 512. 2023-09-09 18:22:05,817 - ControlNet - STATUS - Loading model from cache: control_v11p_sd15_openpose [cab727d4] 2023-09-09 18:22:05,853 - ControlNet - STATUS - Loading preprocessor: openpose_full 2023-09-09 18:22:05,854 - ControlNet - STATUS - preprocessor resolution = 768 2023-09-09 18:22:07,464 - ControlNet - STATUS - ControlNet Hooked - Time = 1.6531829833984375 Tile 1/4 Tile 2/4 Tile 3/4 Tile 4/4 100%|████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:03<00:00, 1.60it/s] [-] ADetailer: applied 1st ad_prompt: 'photo of photo of photo of a woman, green_eyes, in a action movie, green_eyes, in a space capsule, blue_eyes, in a space capsule' [-] ADetailer: applied 1st ad_negative_prompt: 'FastNegativeV2, ugly, ugly' 2023-09-09 18:22:12,691 - ControlNet - STATUS - Loading preprocessor: revision_clipvision 2023-09-09 18:22:12,692 - ControlNet - STATUS - preprocessor resolution = 768 2023-09-09 18:22:12,705 - ControlNet - STATUS - ControlNet Hooked - Time = 0.19816899299621582 2023-09-09 18:22:12,953 - ControlNet - STATUS - Loading preprocessor: revision_clipvision 2023-09-09 18:22:12,953 - ControlNet - STATUS - preprocessor resolution = 768 2023-09-09 18:22:12,970 - ControlNet - STATUS - ControlNet Hooked - Time = 0.02399921417236328 Tile 1/15 Tile 2/15 Tile 3/15 Tile 4/15 Tile 5/15 Tile 6/15 Tile 7/15 Tile 8/15 Tile 9/15 Tile 10/15 Tile 11/15 Tile 12/15 Tile 13/15 Tile 14/15 Tile 15/15 100%|██████████████████████████████████████████████████████████████████████████████████| 11/11 [00:04<00:00, 2.24it/s] mediapipe: 1 detected. | 33/5434 [00:32<47:07, 1.91it/s] 2023-09-09 18:22:19,827 - ControlNet - WARNING - [openpose_full.processor_res] Invalid value(-1), using default value 512. 2023-09-09 18:22:19,827 - ControlNet - STATUS - Loading model from cache: control_v11p_sd15_openpose [cab727d4] 2023-09-09 18:22:19,861 - ControlNet - STATUS - Loading preprocessor: openpose_full 2023-09-09 18:22:19,861 - ControlNet - STATUS - preprocessor resolution = 768 2023-09-09 18:22:21,368 - ControlNet - STATUS - ControlNet Hooked - Time = 1.5476295948028564 Tile 1/4 Tile 2/4 Tile 3/4 Tile 4/4 100%|████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:03<00:00, 1.57it/s] [-] ADetailer: applied 1st ad_prompt: 'photo of photo of photo of photo of a woman, green_eyes, in a action movie, green_eyes, in a horror movie, blue_eyes, in a action movie, blue_eyes, in a space capsule' [-] ADetailer: applied 1st ad_negative_prompt: 'FastNegativeV2, ugly, ugly' 2023-09-09 18:22:26,683 - ControlNet - STATUS - Loading preprocessor: revision_clipvision 2023-09-09 18:22:26,683 - ControlNet - STATUS - preprocessor resolution = 768 2023-09-09 18:22:26,701 - ControlNet - STATUS - ControlNet Hooked - Time = 0.2024543285369873 2023-09-09 18:22:26,961 - ControlNet - STATUS - Loading preprocessor: revision_clipvision 2023-09-09 18:22:26,961 - ControlNet - STATUS - preprocessor resolution = 768 2023-09-09 18:22:26,974 - ControlNet - STATUS - ControlNet Hooked - Time = 0.020000934600830078 Tile 1/15 Tile 2/15 Tile 3/15 Tile 4/15 Tile 5/15 Tile 6/15 Tile 7/15 Tile 8/15 Tile 9/15 Tile 10/15 Tile 11/15 Tile 12/15 Tile 13/15 Tile 14/15 Tile 15/15 82%|███████████████████████████████████████████████████████████████████▉ | 9/11 [00:04<00:00, 2.18it/s] 2023-09-09 18:22:32,709 - ControlNet - STATUS - Loading preprocessor: revision_clipvision/5434 [00:45<54:57, 1.64it/s] 2023-09-09 18:22:32,709 - ControlNet - STATUS - preprocessor resolution = 768 2023-09-09 18:22:32,722 - ControlNet - STATUS - ControlNet Hooked - Time = 0.021003007888793945 Total progress: 1%|▍ | 42/5434 [00:46<1:38:40, 1.10s/it] {"prompt": "photo of photo of photo of photo of a woman, green_eyes, in a horror movie, green_eyes, in a scifi movie, brown_eyes, in a horror movie, green_eyes, in a space capsule", "all_prompts": ["photo of photo of photo of photo of a woman, green_eyes, in a horror movie, green_eyes, in a scifi movie, brown_eyes, in a horror movie, green_eyes, in a space capsule"], "negative_prompt": "FastNegativeV2, ugly", "all_negative_prompts": ["FastNegativeV2, ugly"], "seed": 3882599693, "all_seeds": [3882599693], "subseed": -1, "all_subseeds": [2953649038], "subseed_strength": 0, "width": 768, "height": 1152, "sampler_name": "Euler a", "cfg_scale": 7, "steps": 34, "batch_size": 1, "restore_faces": false, "face_restoration_model": null, "sd_model_name": "icbinpICantBelieveIts_seco", "sd_model_hash": "fa1224c923", "sd_vae_name": null, "sd_vae_hash": null, "seed_resize_from_w": -1, "seed_resize_from_h": -1, "denoising_strength": 0.3, "extra_generation_params": {"ADetailer model": "mediapipe_face_full", "ADetailer confidence": 0.3, "ADetailer dilate/erode": 0, "ADetailer mask blur": 4, "ADetailer denoising strength": 0.29, "ADetailer inpaint only masked": true, "ADetailer inpaint padding": 32, "ADetailer use separate steps": true, "ADetailer steps": 20, "ADetailer ControlNet model": "control_v11p_sd15_openpose [cab727d4]", "ADetailer version": "23.9.1", "ControlNet 0": "Module: revision_clipvision, Model: None, Weight: 1, Resize Mode: Crop and Resize, Low Vram: False, Threshold A: 0, Guidance Start: 0, Guidance End: 1, Pixel Perfect: True, Control Mode: My prompt is more important"}, "index_of_first_image": 0, "infotexts": [""], "styles": ["testing adetailer"], "job_timestamp": "20230909182144", "clip_skip": 1, "is_using_inpainting_conditioning": false, "version": "v1.6.0-124-g92464233"}

akx commented 9 months ago

This may be a conflict with ADetailer. If you entirely disable the ADetailer extension, are you still seeing the same issue?

chuckkay commented 9 months ago

i thought the same, but no if i disable adetailer same problem... but only when the prompt is in the styles format, if i copy the the dynamic prompt from style and x out the style the problem is gone, but when using style the problem comes back

ytakio commented 4 months ago

My PR may fixe it. Current code seems inherit prompts style extracted to the next batch loop. In the next batch loop, inherited prompts will be used with extracting style again. My PR may prevent from inheriting style extracted prompts to next loop. https://github.com/adieyal/sd-dynamic-prompts/pull/712