AUTOMATIC1111 / stable-diffusion-webui

Stable Diffusion web UI
GNU Affero General Public License v3.0
139.62k stars 26.47k forks source link

[Bug]: Repetition of TI info in metadata and a possible solution #13709

Open chromesun opened 11 months ago

chromesun commented 11 months ago

Is there an existing issue for this?

What happened?

When I use one or more TI embeddings in a prompt (or negative prompt) and set Batch count >1, the metadata in generated PNG files contains repeated blocks of the TI(s). e.g. TI hashes: "xlmrblng15-1300: efd052d36da0, xlmrblng15-1300: efd052d36da0" where it should just be TI hashes: "xlmrblng15-1300: efd052d36da0" Weirdly, the TI info is in twice for the first image in the batch, but is in four times for images 2, 3, 4, etc. When I also use the extension Dynamic Prompts, the problem is exacerbated with repetitions increasing linearly as Batch count increases. Not a show-stopping bug, but when your image metadate becomes 100s of lines long (after using multiple TIs in a batch of 100) it's a pain.

I've mentioned this in #13117 in "Additional information" with a screen snippet but I now doubt the two issues are related.

I had a look at the file webui\modules\sd_hijack_clip.py and discovered that commenting out lines 248,249 made the problem go away.

247 if hashes: 248 # if self.hijack.extra_generation_params.get("TI hashes"): 249 # hashes.append(self.hijack.extra_generation_params.get("TI hashes")) 250 self.hijack.extra_generation_params["TI hashes"] = ", ".join(hashes)

It looks to me as if the append is unneccesary, but I know next to nothing about the a1111 webui coding... and Python in general! Does anyone want to weigh in with an opinion? Please :-) I've been using the webui for a couple of hours with my hacked sd_hijack_clip.py and have not noticed any fall-out so far.

Steps to reproduce the problem

Enable "Add Textual Inversion hashes to infotext" in Settings\Extra Networks Create a prompt that uses a TI Set Batch count >1 Generate Metadata shows duplicated TI info.

What should have happened?

Metadata on-screen and in PNG files should show one reference to each TI.

Sysinfo

{ "Platform": "Windows-10-10.0.19045-SP0", "Python": "3.10.7", "Version": "v1.6.0", "Commit": "5ef669de080814067961f28357256e8fe27544f4", "Script path": "G:\stable-diffusion-webui-master\webui", "Data path": "G:\stable-diffusion-webui-master\webui", "Extensions dir": "G:\stable-diffusion-webui-master\webui\extensions", "Checksum": "75e13f7921ae7a41af90fc3a834bf2168ffc7a54d34ccac4b9edc8e1909ee460", "Commandline": [ "launch.py", "--medvram", "--xformers", "--no-half-vae", "--listen", "--api", "--disable-safe-unpickle", "--ckpt", "G:\stable-diffusion-webui-master\webui\models\Stable-diffusion\SDXL\crystalClearXL_ccxl.safetensors" ], "Torch env info": { "torch_version": "2.0.1+cu118", "is_debug_build": "False", "cuda_compiled_version": "11.8", "gcc_version": null, "clang_version": null, "cmake_version": null, "os": "Microsoft Windows 10 Pro", "libc_version": "N/A", "python_version": "3.10.7 (tags/v3.10.7:6cc6b13, Sep 5 2022, 14:08:36) [MSC v.1933 64 bit (AMD64)] (64-bit runtime)", "python_platform": "Windows-10-10.0.19045-SP0", "is_cuda_available": "True", "cuda_runtime_version": null, "cuda_module_loading": "LAZY", "nvidia_driver_version": "536.99", "nvidia_gpu_models": "GPU 0: NVIDIA GeForce RTX 3060", "cudnn_version": null, "pip_version": "pip3", "pip_packages": [ "numpy==1.23.5", "open-clip-torch==2.20.0", "pytorch-lightning==1.9.4", "torch==2.0.1+cu118", "torchdiffeq==0.2.3", "torchmetrics==1.0.3", "torchsde==0.2.5", "torchvision==0.15.2+cu118" ], "conda_packages": null, "hip_compiled_version": "N/A", "hip_runtime_version": "N/A", "miopen_runtime_version": "N/A", "caching_allocator_config": "", "is_xnnpack_available": "True", "cpu_info": [ "Architecture=9", "CurrentClockSpeed=3200", "DeviceID=CPU0", "Family=107", "L2CacheSize=4096", "L2CacheSpeed=", "Manufacturer=AuthenticAMD", "MaxClockSpeed=3200", "Name=AMD Ryzen 7 2700 Eight-Core Processor ", "ProcessorType=3", "Revision=2050" ] }, "Exceptions": [], "CPU": { "model": "AMD64 Family 23 Model 8 Stepping 2, AuthenticAMD", "count logical": 16, "count physical": 8 }, "RAM": { "total": "64GB", "used": "31GB", "free": "33GB" }, "Extensions": [ { "name": "sd-dynamic-prompts", "path": "G:\stable-diffusion-webui-master\webui\extensions\sd-dynamic-prompts", "version": "39c06b30", "branch": "main", "remote": "https://github.com/adieyal/sd-dynamic-prompts.git" } ], "Inactive extensions": [ { "name": "DreamArtist-sd-webui-extension", "path": "G:\stable-diffusion-webui-master\webui\extensions\DreamArtist-sd-webui-extension", "version": "12f80775", "branch": "master", "remote": "https://github.com/7eu7d7/DreamArtist-sd-webui-extension.git" }, { "name": "SD-latent-mirroring", "path": "G:\stable-diffusion-webui-master\webui\extensions\SD-latent-mirroring", "version": "05df9e83", "branch": "main", "remote": "https://github.com/dfaker/SD-latent-mirroring.git" }, { "name": "clip-interrogator-ext", "path": "G:\stable-diffusion-webui-master\webui\extensions\clip-interrogator-ext", "version": "0f1a4591", "branch": "main", "remote": "https://github.com/pharmapsychotic/clip-interrogator-ext.git" }, { "name": "multidiffusion-upscaler-for-automatic1111", "path": "G:\stable-diffusion-webui-master\webui\extensions\multidiffusion-upscaler-for-automatic1111", "version": "f9f8073e", "branch": "main", "remote": "https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111.git" }, { "name": "sd-dynamic-thresholding", "path": "G:\stable-diffusion-webui-master\webui\extensions\sd-dynamic-thresholding", "version": "55ca687f", "branch": "master", "remote": "https://github.com/mcmonkeyprojects/sd-dynamic-thresholding.git" }, { "name": "sd-webui-controlnet", "path": "G:\stable-diffusion-webui-master\webui\extensions\sd-webui-controlnet", "version": "e67e0177", "branch": "main", "remote": "https://github.com/Mikubill/sd-webui-controlnet.git" }, { "name": "stable-diffusion-webui-aesthetic-gradients", "path": "G:\stable-diffusion-webui-master\webui\extensions\stable-diffusion-webui-aesthetic-gradients", "version": "2624e5dd", "branch": "master", "remote": "https://github.com/AUTOMATIC1111/stable-diffusion-webui-aesthetic-gradients.git" }, { "name": "ultimate-upscale-for-automatic1111", "path": "G:\stable-diffusion-webui-master\webui\extensions\ultimate-upscale-for-automatic1111", "version": "728ffcec", "branch": "master", "remote": "https://github.com/Coyote-A/ultimate-upscale-for-automatic1111.git" } ], "Environment": { "COMMANDLINE_ARGS": "--medvram --xformers --no-half-vae --listen --api --disable-safe-unpickle --ckpt \"G:\stable-diffusion-webui-master\webui\models\Stable-diffusion\SDXL\crystalClearXL_ccxl.safetensors\"", "GRADIO_ANALYTICS_ENABLED": "False" }, "Config": { "samples_save": true, "samples_format": "png", "samples_filenamepattern": "[datetime<%Y%m%d%H%M%S>] [prompt_spaces] [seed] c=[cfg] [steps] [sampler] [model_name]", "save_images_add_number": false, "grid_save": true, "grid_format": "png", "grid_extended_filename": true, "grid_only_if_multiple": true, "grid_prevent_empty_spots": false, "grid_zip_filename_pattern": "", "n_rows": -1, "font": "", "grid_text_active_color": "#000000", "grid_text_inactive_color": "#999999", "grid_background_color": "#ffffff", "enable_pnginfo": true, "save_txt": false, "save_images_before_face_restoration": false, "save_images_before_highres_fix": false, "save_images_before_color_correction": false, "save_mask": false, "save_mask_composite": false, "jpeg_quality": 80, "webp_lossless": false, "export_for_4chan": false, "img_downscale_threshold": 4.0, "target_side_length": 4000, "img_max_size_mp": 200, "use_original_name_batch": true, "use_upscaler_name_as_suffix": false, "save_selected_only": true, "save_init_img": false, "temp_dir": "", "clean_temp_dir_at_start": false, "outdir_samples": "", "outdir_txt2img_samples": "Q:\AI\SDXL\3060\txt2img", "outdir_img2img_samples": "Q:\AI\SDXL\3060\img2img", "outdir_extras_samples": "Q:\AI\SDXL\3060\extras", "outdir_grids": "", "outdir_txt2img_grids": "Q:\AI\SDXL\3060\txt2img-grids", "outdir_img2img_grids": "Q:\AI\SDXL\3060\img2img-grids", "outdir_save": "log/images", "outdir_init_images": "outputs/init-images", "save_to_dirs": false, "grid_save_to_dirs": false, "use_save_to_dirs_for_ui": false, "directories_filename_pattern": "[date]", "directories_max_prompt_words": 8, "ESRGAN_tile": 192, "ESRGAN_tile_overlap": 8, "realesrgan_enabled_models": [ "R-ESRGAN 4x+", "R-ESRGAN General 4xV3", "R-ESRGAN General WDN 4xV3", "R-ESRGAN 2x+" ], "upscaler_for_img2img": "R-ESRGAN 4x+", "face_restoration_model": "GFPGAN", "code_former_weight": 0.5, "face_restoration_unload": true, "show_warnings": false, "memmon_poll_rate": 8, "samples_log_stdout": false, "multiple_tqdm": true, "print_hypernet_extra": false, "list_hidden_files": true, "disable_mmap_load_safetensors": false, "unload_models_when_training": false, "pin_memory": false, "save_optimizer_state": true, "save_training_settings_to_txt": true, "dataset_filename_word_regex": "", "dataset_filename_join_string": " ", "training_image_repeats_per_epoch": 1, "training_write_csv_every": 25.0, "training_xattention_optimizations": false, "training_enable_tensorboard": false, "training_tensorboard_save_images": false, "training_tensorboard_flush_every": 120, "sd_model_checkpoint": "SDXL\crystalClearXL_ccxl.safetensors [0b76532e03]", "sd_checkpoint_cache": 0, "sd_vae_checkpoint_cache": 0, "sd_vae": "sdxl_vae.safetensors", "sd_vae_as_default": true, "sd_unet": "Automatic", "inpainting_mask_weight": 1.0, "initial_noise_multiplier": 1.0, "img2img_color_correction": false, "img2img_fix_steps": false, "img2img_background_color": "#ffffff", "enable_quantization": true, "enable_emphasis": true, "enable_batch_seeds": true, "comma_padding_backtrack": 20, "CLIP_stop_at_last_layers": 1, "upcast_attn": false, "auto_vae_precision": true, "randn_source": "GPU", "sdxl_crop_top": 0, "sdxl_crop_left": 0, "sdxl_refiner_low_aesthetic_score": 2.5, "sdxl_refiner_high_aesthetic_score": 6.0, "cross_attention_optimization": "Automatic", "s_min_uncond": 1, "token_merging_ratio": 0.0, "token_merging_ratio_img2img": 0.0, "token_merging_ratio_hr": 0.0, "pad_cond_uncond": false, "experimental_persistent_cond_cache": false, "use_old_emphasis_implementation": false, "use_old_karras_scheduler_sigmas": false, "no_dpmpp_sde_batch_determinism": false, "use_old_hires_fix_width_height": false, "dont_fix_second_order_samplers_schedule": false, "hires_fix_use_firstpass_conds": false, "interrogate_keep_models_in_memory": false, "interrogate_return_ranks": false, "interrogate_clip_num_beams": 1, "interrogate_clip_min_length": 24, "interrogate_clip_max_length": 48, "interrogate_clip_dict_limit": 1500, "interrogate_clip_skip_categories": [], "interrogate_deepbooru_score_threshold": 0.5, "deepbooru_sort_alpha": true, "deepbooru_use_spaces": true, "deepbooru_escape": true, "deepbooru_filter_tags": "", "extra_networks_show_hidden_directories": true, "extra_networks_hidden_models": "When searched", "extra_networks_default_multiplier": 1.0, "extra_networks_card_width": 0, "extra_networks_card_height": 0, "extra_networks_card_text_scale": 1.0, "extra_networks_card_show_desc": true, "extra_networks_add_text_separator": " ", "ui_extra_networks_tab_reorder": "", "textual_inversion_print_at_load": false, "textual_inversion_add_hashes_to_infotext": true, "sd_hypernetwork": "None", "localization": "None", "gradio_theme": "Default", "img2img_editor_height": 720, "return_grid": true, "return_mask": false, "return_mask_composite": false, "do_not_show_images": false, "send_seed": true, "send_size": true, "js_modal_lightbox": true, "js_modal_lightbox_initially_zoomed": true, "js_modal_lightbox_gamepad": false, "js_modal_lightbox_gamepad_repeat": 250, "show_progress_in_title": true, "samplers_in_dropdown": true, "dimensions_and_batch_together": true, "keyedit_precision_attention": 0.1, "keyedit_precision_extra": 0.05, "keyedit_delimiters": ".,\/!?%^*;:{}=`~()", "keyedit_move": true, "quicksettings_list": [ "sd_model_checkpoint", "CLIP_stop_at_last_layers", "sd_vae", "sd_hypernetwork" ], "ui_tab_order": [], "hidden_tabs": [], "ui_reorder_list": [ "sampler", "dimensions", "cfg", "seed", "checkboxes", "hires_fix", "batch", "scripts" ], "hires_fix_show_sampler": false, "hires_fix_show_prompts": false, "disable_token_counters": false, "add_model_hash_to_info": true, "add_model_name_to_info": true, "add_user_name_to_info": false, "add_version_to_infotext": false, "disable_weights_auto_swap": true, "infotext_styles": "Apply if any", "show_progressbar": true, "live_previews_enable": true, "live_previews_image_format": "png", "show_progress_grid": true, "show_progress_every_n_steps": 10, "show_progress_type": "Approx NN", "live_preview_content": "Prompt", "live_preview_refresh_period": 1000, "hide_samplers": [], "eta_ddim": 0.0, "eta_ancestral": 1.0, "ddim_discretize": "uniform", "s_churn": 0.0, "s_tmin": 0.0, "s_noise": 1.0, "k_sched_type": "Automatic", "sigma_min": 0.0, "sigma_max": 0.0, "rho": 0.0, "eta_noise_seed_delta": 31337, "always_discard_next_to_last_sigma": false, "uni_pc_variant": "bh1", "uni_pc_skip_type": "time_uniform", "uni_pc_order": 3, "uni_pc_lower_order_final": true, "postprocessing_enable_in_main_ui": [], "postprocessing_operation_order": [], "upscaling_max_images_in_cache": 5, "disabled_extensions": [ "DreamArtist-sd-webui-extension", "SD-latent-mirroring", "clip-interrogator-ext", "multidiffusion-upscaler-for-automatic1111", "sd-dynamic-thresholding", "sd-webui-controlnet", "stable-diffusion-webui-aesthetic-gradients", "ultimate-upscale-for-automatic1111" ], "disable_all_extensions": "none", "restore_config_state_file": "", "sd_checkpoint_hash": "0b76532e03a1bac388cbf559af00384abcbd2b5b3f8834158ae4b1b9146a3843", "ldsr_steps": 100, "ldsr_cached": false, "SCUNET_tile": 256, "SCUNET_tile_overlap": 8, "SWIN_tile": 192, "SWIN_tile_overlap": 8, "lora_functional": false, "sd_lora": "None", "lora_preferred_name": "Alias from file", "lora_add_hashes_to_infotext": true, "lora_show_all": false, "lora_hide_unknown_for_versions": [], "extra_options": [], "extra_options_accordion": false, "canvas_hotkey_zoom": "Alt", "canvas_hotkey_adjust": "Ctrl", "canvas_hotkey_move": "F", "canvas_hotkey_fullscreen": "S", "canvas_hotkey_reset": "R", "canvas_hotkey_overlap": "O", "canvas_show_tooltip": true, "canvas_blur_prompt": false, "canvas_disabled_functions": [ "Overlap" ], "control_net_model_config": "models\cldm_v15.yaml", "control_net_model_adapter_config": "models\t2iadapter_sketch_sd14v1.yaml", "control_net_detectedmap_dir": "detected_maps", "control_net_models_path": "", "control_net_modules_path": "", "control_net_max_models_num": 3, "control_net_model_cache_size": 1, "control_net_inpaint_blur_sigma": 7, "control_net_no_high_res_fix": false, "control_net_no_detectmap": false, "control_net_detectmap_autosaving": false, "control_net_allow_script_control": false, "control_net_sync_field_args": false, "controlnet_show_batch_images_in_ui": false, "controlnet_increment_seed_during_batch": false, "controlnet_disable_control_type": false, "controlnet_disable_openpose_edit": false, "dp_ignore_whitespace": false, "dp_write_raw_template": true, "dp_write_prompts_to_file": true, "dp_parser_variant_start": "{", "dp_parser_variant_end": "}", "dp_parser_wildcard_wrap": "__", "dp_limit_jinja_prompts": false, "dp_auto_purge_cache": false, "dp_wildcard_manager_no_dedupe": false, "dp_wildcard_manager_no_sort": false, "dp_wildcard_manager_shuffle": false, "dp_magicprompt_default_model": "Gustavosta/MagicPrompt-Stable-Diffusion", "dp_magicprompt_batch_size": 1, "controlnet_ignore_noninpaint_mask": false, "sd_vae_overrides_per_model_preferences": false, "save_incomplete_images": false, "face_restoration": false, "auto_launch_browser": "Disable", "show_gradio_deprecation_warnings": true, "hide_ldm_prints": true, "api_enable_requests": true, "api_forbid_local_requests": true, "api_useragent": "", "sd_checkpoints_limit": 1, "sd_checkpoints_keep_in_cpu": true, "tiling": false, "sd_vae_encode_method": "Full", "sd_vae_decode_method": "Full", "img2img_extra_noise": 0, "img2img_sketch_default_brush_color": "#ffffff", "img2img_inpaint_mask_brush_color": "#ffffff", "img2img_inpaint_sketch_default_brush_color": "#ffffff", "persistent_cond_cache": true, "batch_cond_uncond": true, "use_old_scheduling": false, "lora_in_memory_limit": 0, "gradio_themes_cache": true, "gallery_height": "", "extra_options_txt2img": [], "extra_options_img2img": [], "extra_options_cols": 1, "live_preview_allow_lowvram_full": false, "live_preview_fast_interrupt": false, "s_tmax": 0, "canvas_auto_expand": true, "sgm_noise_multiplier": false, "hires_fix_refiner_pass": "second pass", "control_net_unit_count": 3 }, "Startup": { "total": 54.13893723487854, "records": { "initial startup": 0.0030028820037841797, "prepare environment/checks": 0.05405020713806152, "prepare environment/git version info": 0.2822563648223877, "prepare environment/torch GPU test": 5.962759733200073, "prepare environment/clone repositores": 0.9398574829101562, "prepare environment/run extensions installers/sd-dynamic-prompts": 0.7937238216400146, "prepare environment/run extensions installers": 0.7937238216400146, "prepare environment": 8.341928482055664, "launcher": 0.011011362075805664, "import torch": 14.758470296859741, "import gradio": 3.884272813796997, "setup paths": 3.354063034057617, "import ldm": 0.03402543067932129, "import sgm": 0.0, "initialize shared": 0.8067352771759033, "other imports": 2.8445944786071777, "opts onchange": 0.0010020732879638672, "setup SD model": 0.021016597747802734, "setup codeformer": 0.6986370086669922, "setup gfpgan": 0.13212060928344727, "set samplers": 0.0, "list extensions": 0.009008169174194336, "restore config state file": 0.0, "list SD models": 0.09308481216430664, "list localizations": 0.003002643585205078, "load scripts/custom_code.py": 0.020020723342895508, "load scripts/img2imgalt.py": 0.0019998550415039062, "load scripts/loopback.py": 0.002001523971557617, "load scripts/outpainting_mk_2.py": 0.0030028820037841797, "load scripts/poor_mans_outpainting.py": 0.002001047134399414, "load scripts/postprocessing_codeformer.py": 0.0010008811950683594, "load scripts/postprocessing_gfpgan.py": 0.0020024776458740234, "load scripts/postprocessing_upscale.py": 0.0020012855529785156, "load scripts/prompt_matrix.py": 0.002002716064453125, "load scripts/prompts_from_file.py": 0.0009999275207519531, "load scripts/refiner.py": 0.0020029544830322266, "load scripts/sd_upscale.py": 0.0020020008087158203, "load scripts/seed.py": 0.0020017623901367188, "load scripts/xyz_grid.py": 0.004003047943115234, "load scripts/dynamic_prompting.py": 0.21719813346862793, "load scripts/ldsr_model.py": 12.764574527740479, "load scripts/lora_script.py": 0.4213840961456299, "load scripts/scunet_model.py": 0.07306623458862305, "load scripts/swinir_model.py": 0.06005740165710449, "load scripts/hotkey_config.py": 0.001999378204345703, "load scripts/extra_options_section.py": 0.0010008811950683594, "load scripts": 13.586323738098145, "load upscalers": 0.05104660987854004, "refresh VAE": 0.014012813568115234, "refresh textual inversion templates": 0.0010001659393310547, "scripts list_optimizers": 0.009009122848510742, "scripts list_unets": 0.0, "reload hypernetworks": 0.01303553581237793, "initialize extra networks": 0.03600907325744629, "scripts before_ui_callback": 0.004003763198852539, "create ui": 0.9678816795349121, "gradio launch": 4.624785661697388, "add APIs": 0.14513397216796875, "app_started_callback/lora_script.py": 0.0, "app_started_callback": 0.0 } }, "Packages": [ "absl-py==1.4.0", "accelerate==0.21.0", "addict==2.4.0", "aenum==3.1.15", "aiofiles==23.2.1", "aiohttp==3.8.5", "aiosignal==1.3.1", "altair==5.0.1", "antlr4-python3-runtime==4.9.3", "anyio==3.7.1", "async-timeout==4.0.2", "attrs==23.1.0", "basicsr==1.4.2", "beautifulsoup4==4.12.2", "blendmodes==2022", "blip-ci==0.0.5", "boltons==23.0.0", "cachetools==5.3.1", "certifi==2023.7.22", "cffi==1.15.1", "charset-normalizer==3.2.0", "clean-fid==0.1.35", "click==8.1.6", "clip-interrogator==0.6.0", "clip==1.0", "colorama==0.4.6", "contourpy==1.1.0", "cssselect2==0.7.0", "cycler==0.11.0", "deprecation==2.1.0", "dynamicprompts==0.29.0", "einops==0.4.1", "exceptiongroup==1.1.2", "facexlib==0.3.0", "fairscale==0.4.4", "fastapi==0.94.0", "ffmpy==0.3.1", "filelock==3.12.2", "filterpy==1.4.5", "flatbuffers==23.5.26", "fonttools==4.42.0", "frozenlist==1.4.0", "fsspec==2023.6.0", "ftfy==6.1.1", "future==0.18.3", "fvcore==0.1.5.post20221221", "gdown==4.7.1", "gfpgan==1.3.8", "gitdb==4.0.10", "gitpython==3.1.32", "google-auth-oauthlib==1.0.0", "google-auth==2.22.0", "gradio-client==0.5.0", "gradio==3.41.2", "grpcio==1.56.2", "h11==0.12.0", "httpcore==0.15.0", "httpx==0.24.1", "huggingface-hub==0.16.4", "idna==3.4", "imageio==2.31.1", "importlib-metadata==6.8.0", "importlib-resources==6.0.1", "inflection==0.5.1", "iopath==0.1.9", "jinja2==3.1.2", "joblib==1.3.2", "jsonmerge==1.8.0", "jsonschema-specifications==2023.7.1", "jsonschema==4.19.0", "kiwisolver==1.4.4", "kornia==0.6.7", "lark==1.1.2", "lazy-loader==0.3", "lightning-utilities==0.9.0", "linkify-it-py==2.0.2", "llvmlite==0.40.1", "lmdb==1.4.1", "lpips==0.1.4", "lxml==4.9.3", "markdown-it-py==2.2.0", "markdown==3.4.4", "markupsafe==2.1.3", "matplotlib==3.7.2", "mdit-py-plugins==0.3.3", "mdurl==0.1.2", "mediapipe==0.10.3", "mpmath==1.3.0", "multidict==6.0.4", "networkx==3.1", "numba==0.57.1", "numpy==1.23.5", "oauthlib==3.2.2", "omegaconf==2.2.3", "open-clip-torch==2.20.0", "opencv-contrib-python==4.8.0.76", "opencv-python==4.8.0.76", "orjson==3.9.4", "packaging==23.1", "pandas==2.0.3", "piexif==1.1.3", "pillow==9.5.0", "pip==22.2.2", "platformdirs==3.10.0", "portalocker==2.7.0", "protobuf==3.20.0", "psutil==5.9.5", "pyasn1-modules==0.3.0", "pyasn1==0.5.0", "pycparser==2.21", "pydantic==1.10.12", "pydub==0.25.1", "pygments==2.16.1", "pyparsing==3.0.9", "pysocks==1.7.1", "python-dateutil==2.8.2", "python-multipart==0.0.6", "pytorch-lightning==1.9.4", "pytz==2023.3", "pywavelets==1.4.1", "pywin32==306", "pyyaml==6.0.1", "realesrgan==0.3.0", "referencing==0.30.2", "regex==2023.8.8", "reportlab==4.0.4", "requests-oauthlib==1.3.1", "requests==2.31.0", "resize-right==0.0.2", "rpds-py==0.9.2", "rsa==4.9", "safetensors==0.3.1", "scikit-image==0.21.0", "scikit-learn==1.3.0", "scipy==1.11.1", "semantic-version==2.10.0", "send2trash==1.8.2", "sentencepiece==0.1.99", "setuptools==63.2.0", "six==1.16.0", "smmap==5.0.0", "sniffio==1.3.0", "sounddevice==0.4.6", "soupsieve==2.4.1", "starlette==0.26.1", "svglib==1.5.1", "sympy==1.12", "tabulate==0.9.0", "tb-nightly==2.14.0a20230808", "tensorboard-data-server==0.7.1", "termcolor==2.3.0", "threadpoolctl==3.2.0", "tifffile==2023.7.18", "timm==0.9.2", "tinycss2==1.2.1", "tokenizers==0.13.3", "tomesd==0.1.3", "tomli==2.0.1", "toolz==0.12.0", "torch==2.0.1+cu118", "torchdiffeq==0.2.3", "torchmetrics==1.0.3", "torchsde==0.2.5", "torchvision==0.15.2+cu118", "tqdm==4.66.0", "trampoline==0.1.2", "transformers==4.30.2", "typing-extensions==4.7.1", "tzdata==2023.3", "uc-micro-py==1.0.2", "urllib3==1.26.16", "uvicorn==0.23.2", "wcwidth==0.2.6", "webencodings==0.5.1", "websockets==11.0.3", "werkzeug==2.3.6", "wheel==0.41.1", "xformers==0.0.20", "yacs==0.1.8", "yapf==0.40.1", "yarl==1.9.2", "zipp==3.16.2" ] }

What browsers do you use to access the UI ?

Mozilla Firefox, Microsoft Edge

Console logs

venv "G:\stable-diffusion-webui-master\webui\venv\Scripts\Python.exe"
Python 3.10.7 (tags/v3.10.7:6cc6b13, Sep  5 2022, 14:08:36) [MSC v.1933 64 bit (AMD64)]
Version: v1.6.0
Commit hash: 5ef669de080814067961f28357256e8fe27544f4
Launching Web UI with arguments: --medvram --xformers --no-half-vae --listen --api --disable-safe-unpickle --ckpt G:\stable-diffusion-webui-master\webui\models\Stable-diffusion\SDXL\crystalClearXL_ccxl.safetensors
Loading weights [0b76532e03] from G:\stable-diffusion-webui-master\webui\models\Stable-diffusion\SDXL\crystalClearXL_ccxl.safetensors
Running on local URL:  http://0.0.0.0:7860
Creating model from config: G:\stable-diffusion-webui-master\webui\repositories\generative-models\configs\inference\sd_xl_base.yaml

To create a public link, set `share=True` in `launch()`.
Startup time: 54.1s (prepare environment: 8.3s, import torch: 14.8s, import gradio: 3.9s, setup paths: 3.4s, initialize shared: 0.8s, other imports: 2.8s, setup codeformer: 0.7s, setup gfpgan: 0.1s, load scripts: 13.6s, create ui: 1.0s, gradio launch: 4.6s, add APIs: 0.1s).
Loading VAE weights specified in settings: G:\stable-diffusion-webui-master\webui\models\VAE\sdxl_vae.safetensors
Applying attention optimization: xformers... done.
Model loaded in 11.6s (load weights from disk: 2.1s, create model: 0.7s, apply weights to model: 3.1s, apply half(): 0.1s, load VAE: 0.2s, load textual inversion embeddings: 1.4s, calculate empty prompt: 3.9s).
INFO:sd_dynamic_prompts.dynamic_prompting:Prompt matrix will create 4 images in a total of 4 batches.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 40/40 [00:22<00:00,  1.76it/s]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 40/40 [00:19<00:00,  2.08it/s]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 40/40 [00:19<00:00,  2.07it/s]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 40/40 [00:19<00:00,  2.08it/s]
Total progress: 100%|███████████████████████████████████████████████████████████████████████████████████| 160/160 [01:30<00:00,  1.77it/s]
Total progress: 100%|███████████████████████████████████████████████████████████████████████████████████| 160/160 [01:30<00:00,  2.84it/s]

Additional information

No response

kurayamiblackheart commented 6 months ago

I hope this gets fixed soon

chromesun commented 6 months ago

@kurayamiblackheart I've been using the little hack in webui\modules\sd_hijack_clip.py that I mentioned at top of post since October with no apparent ill effects... just no repetition of TI info. 🙂 (After making a back-up copy of sd_hijack_clip.py) I used Notepad to comment out lines 248, 249 as shown above.

thecakeisal1e commented 6 months ago

Unfortunately, I think it is adding more than just TI info and commenting out those two lines doesn't fix the problem; It seems to be actively duplicating the uses of the embeddings because, if you load a couple embeddings into the negative and then turn your batch size up to 8, it will throw a

RuntimeError: The expanded size of the tensor (##) must match the existing size (##) at non-singleton dimension 0

as if it has added too many tokens to the negative.

chromesun commented 6 months ago

@thecakeisal1e Unfortunately I can't test that - can barely manage a batch size of 2, never mind 8. Also, I don't use negative embeddings in SDXL at all, so if there was general weird behaviour I wouldn't see it. Out of curiousity I just tried a batch count = 8 run with one +ve TI and 2 -ve TIs. The run completed OK and the metadata in the final JPG seems fine with no repetition. Whether webui is actually using the -ve TIs I can't tell.

I can't tell if the effect of the +ve TI is duplicated/doubled because I've nothing to compare it to. However... although I don't use ComfyUI myself, ppl that use my TIs with Comfy say they often have to increase the weighting. I thought that was just a Comfy thing, but perhaps there's more to it? Back in a1111 webui, if I use an 8-vector TI then the webui counter shows +8 tokens as expected. No indication of duplication but as we all know, just 'cos the software says something doesn't make it true 🙄

The parameter block for the 8th image, in the batch run mentioned above, shows this:

cs-cr1stal-v2 solidified smoky quartz zombie horse, silver wire edging Negative prompt: cs-m3tal-ok_v06-1000, dw01-3400 Steps: 40, Sampler: DPM++ 2M Karras, CFG scale: 8, Seed: 3655753132, Size: 1024x1024, Model hash: 0b76532e03, Model: crystalClearXL_ccxl, VAE hash: 235745af8d, VAE: sdxl_vae.safetensors, NGMS: 1, Template: "cs-cr1stal-v2 surface_texture_2 gems_s_2 __MARBLE1/fantasy_mount, chromesun_colours_2__ wire edging", Negative Template: "cs-m3tal-ok_v06-1000, dw01-3400", TI hashes: "cs-cr1stal-v2: 724c80ab7720"

I would have expected to see the two TIs in the -ve prompt listed in the "TI hashes" section. The TIs are all mine and work as expected in the +ve prompt... no idea what they should do in a -ve prompt! In this case they don't appear to affect the image in any drastic way. Normal words in the -ve work as expected, but as I said I've never used a TI in -ve for SDXL.

Running again (same seed) with an empty -ve prompt gives very similar images but stylistically richer. However that could simply be because "cs-m3tal-ok_v06-1000, dw01-3400" is 17 tokens and thus must have some effect on the generation process.

I don't know enough Python to delve any deeper than I have - I recognise that simply commenting out 2 lines is somewhat magical thinking without doing a full trace/review.

In case it's relevant for comparison, I'm using Win10 with a1111 webui v1.7.0 and Dynamic Prompts 284d3ef3 / 2023-12-29. Not up to date I know, but it's a relatively stable set-up for the moment. GPU is 3060/12GB, CPU Ryzen 8c/16t, sysram 64GB. Python is v3.10.9.