search

hako-mikan / sd-webui-regional-prompter

set prompt to divided region
GNU Affero General Public License v3.0
1.58k stars 133 forks source link

Hires Fix cannot be used at the same time with Prompt Mode. #316

Open DavideAlidosi opened 7 months ago

DavideAlidosi commented 7 months ago

Describe the bug Hires Fix cannot be used at the same time with Prompt Mode. To replicate the bug: 1) Activate Regional Prompter 2) Open the Prompt tab 3) Activate Highres Fix 4) Use the following prompt:

a lady ,hair, face BREAK
red ,hair BREAK
tanned ,face

Error returned:

Regional Prompter Active, Pos tokens : [11, 12], Neg tokens : [0]
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:04<00:00,  4.91it/s]
Total progress: 100it [01:16,  1.31it/s]
Error parsing "sv_negative: "  4.97it/s]
Regional Prompter Active, Pos tokens : [11, 12], Neg tokens : [0]
tiled upscale: 100%|███████████████████████████████████████████████████████████████████| 24/24 [00:03<00:00,  7.91it/s]
  0%|                                                                                            | 0/5 [00:00<?, ?it/s]
*** Error completing request
*** Arguments: ('task(5fe1suiflxvq2wf)', <gradio.routes.Request object at 0x0000023DA6319840>, [{'name': 'L:\\WebUI\\webui\\output\\txt2img-images\\2024-04-07\\00590-3526364301.png?1712501946.9005544', 'data': 'http://127.0.0.1:7860/file=L:\\WebUI\\webui\\output\\txt2img-images\\2024-04-07\\00590-3526364301.png?1712501946.9005544', 'is_file': True}, {'name': 'C:\\Users\\indrema\\AppData\\Local\\Temp\\gradio\\tmpncq60b9t.png', 'data': 'http://127.0.0.1:7860/file=C:\\Users\\indrema\\AppData\\Local\\Temp\\gradio\\tmpncq60b9t.png', 'is_file': True}], 0, '{"prompt": "a lady ,hair, face BREAK, red, hair BREAK, tanned ,face", "all_prompts": ["a lady ,hair, face BREAK, red, hair BREAK, tanned ,face"], "negative_prompt": "", "all_negative_prompts": [""], "seed": 3526364301, "all_seeds": [3526364301], "subseed": 1902710801, "all_subseeds": [1902710801], "subseed_strength": 0, "width": 720, "height": 960, "sampler_name": "LMS Karras", "cfg_scale": 6, "steps": 20, "batch_size": 1, "restore_faces": false, "face_restoration_model": null, "sd_model_name": "ultima", "sd_model_hash": "c818ee1217", "sd_vae_name": null, "sd_vae_hash": null, "seed_resize_from_w": -1, "seed_resize_from_h": -1, "denoising_strength": 0.4, "extra_generation_params": {"sv_prompt": "a lady ,hair, face  BREAK\\nred, hair BREAK\\ntanned ,face", "sv_negative": "", "RP Active": true, "RP Divide mode": "Prompt", "RP Matrix submode": "Columns", "RP Mask submode": "Mask", "RP Prompt submode": "Prompt", "RP Calc Mode": "Attention", "RP Ratios": "1;1;4,1,1,1", "RP Base Ratios": "0.2", "RP Use Base": false, "RP Use Common": true, "RP Use Ncommon": false, "RP Options": [false], "RP LoRA Neg Te Ratios": "0", "RP LoRA Neg U Ratios": "0", "RP threshold": "0.4", "RP LoRA Stop Step": "0", "RP LoRA Hires Stop Step": "0", "RP Flip": false, "Hires CFG scale": 3, "Pad conds": true}, "index_of_first_image": 0, "infotexts": ["a lady ,hair, face BREAK, red, hair BREAK, tanned ,face\\nSteps: 20, Sampler: LMS Karras, CFG scale: 6, Seed: 3526364301, Size: 720x960, Model hash: c818ee1217, Model: ultima, Clip skip: 2, RNG: CPU, sv_prompt: \\"a lady ,hair, face  BREAK\\\\nred, hair BREAK\\\\ntanned ,face\\", sv_negative: , RP Active: True, RP Divide mode: Prompt, RP Matrix submode: Columns, RP Mask submode: Mask, RP Prompt submode: Prompt, RP Calc Mode: Attention, RP Ratios: \\"1;1;4,1,1,1\\", RP Base Ratios: 0.2, RP Use Base: False, RP Use Common: True, RP Use Ncommon: False, RP Options: [False], RP LoRA Neg Te Ratios: 0, RP LoRA Neg U Ratios: 0, RP threshold: 0.4, RP LoRA Stop Step: 0, RP LoRA Hires Stop Step: 0, RP Flip: False, Hires CFG scale: 3, Pad conds: True, Version: v1.8.0"], "styles": [], "job_timestamp": "20240407165902", "clip_skip": 2, "is_using_inpainting_conditioning": false, "version": "v1.8.0"}', 'a lady ,hair, face  BREAK\nred, hair BREAK\ntanned ,face', '', [], 20, 'LMS Karras', 1, 1, 6, 960, 720, False, 0.4, 2, '4x_NMKD-Superscale-SP_178000_G', 5, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], 0, False, '', 0.8, -1, False, -1, 0, 0, 0, False, False, {'ad_model': 'face_yolov8n.pt', 'ad_model_classes': '', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, {'ad_model': 'None', 'ad_model_classes': '', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, False, 'MultiDiffusion', False, True, 1024, 1024, 96, 96, 48, 4, 'None', 2, False, 10, 1, 1, 64, False, False, False, False, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 'DemoFusion', False, 128, 64, 4, 2, False, 10, 1, 1, 64, False, True, 3, 1, 1, True, 0.85, 0.6, 4, False, False, 3072, 192, True, True, True, False, True, False, 1, False, False, False, 1.1, 1.5, 100, 0.7, False, True, False, False, False, 0, 'AUTOMATIC/promptgen-lexart', '', UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=-1, threshold_a=-1, threshold_b=-1, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', inpaint_crop_input_image=False, hr_option='Both', save_detected_map=True, advanced_weighting=None), UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=-1, threshold_a=-1, threshold_b=-1, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', inpaint_crop_input_image=False, hr_option='Both', save_detected_map=True, advanced_weighting=None), UiControlNetUnit(enabled=False, module='none', model='None', weight=1, image=None, resize_mode='Crop and Resize', low_vram=False, processor_res=-1, threshold_a=-1, threshold_b=-1, guidance_start=0, guidance_end=1, pixel_perfect=False, control_mode='Balanced', inpaint_crop_input_image=False, hr_option='Both', save_detected_map=True, advanced_weighting=None), 3, False, 'Default', 'Default', 1, False, 0, False, 0, 0, 0, 0, True, False, False, 'ultima.safetensors [c818ee1217]', 'None', 3, '', {'save_settings': ['fp16', 'prune', 'safetensors'], 'calc_settings': ['GPU', 'fastrebasin']}, True, False, False, 'None', 'None', 'None', 'Sum', 'Sum', 'Sum', 0.5, 0.5, 0.5, True, True, True, [], [], [], [], [], [], '0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5', '0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5', '0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5', False, False, False, '', '', '', 'Normal', 'Normal', 'Normal', True, False, 'Prompt', 'Columns', 'Mask', 'Prompt', '1;1;4,1,1,1', '0.2', False, True, False, 'Attention', [False], '0', '0', '0.4', None, '0', '0', False, False, False, 'positive', 'comma', 0, False, False, 'start', '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, False, False, False, 0, False, 5, 'all', 'all', 'all', '', '', 'BREAK $(Negative Prompt)$(Vibrant Colors)', '1', 'none', False, '', '', 'comma', '', True, '', '20', 'all', 'all', 'all', 'all', 0, '', True, '0', False, 'SD1.5', 'All (random)...', 'dynamic', 'superprompt-v1', None, None, False, None, None, False, None, None, False, 50, [], 30, '', 4, [], 1, '', '', '', '') {}
    Traceback (most recent call last):
      File "L:\WebUI\webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "L:\WebUI\webui\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "L:\WebUI\webui\modules\txt2img.py", line 84, in txt2img_upscale
        processed = processing.process_images(p)
      File "L:\WebUI\webui\modules\processing.py", line 785, in process_images
        res = process_images_inner(p)
      File "L:\WebUI\webui\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 59, in processing_process_images_hijack
        return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
      File "L:\WebUI\webui\modules\processing.py", line 921, in process_images_inner
        samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
      File "L:\WebUI\webui\modules\processing.py", line 1273, in sample
        return self.sample_hr_pass(samples, decoded_samples, seeds, subseeds, subseed_strength, prompts)
      File "L:\WebUI\webui\modules\processing.py", line 1358, in sample_hr_pass
        samples = self.sampler.sample_img2img(self, samples, noise, self.hr_c, self.hr_uc, steps=self.hr_second_pass_steps or self.steps, image_conditioning=image_conditioning)
      File "L:\WebUI\webui\modules\sd_samplers_kdiffusion.py", line 188, in sample_img2img
        samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "L:\WebUI\webui\modules\sd_samplers_common.py", line 261, in launch_sampling
        return func()
      File "L:\WebUI\webui\modules\sd_samplers_kdiffusion.py", line 188, in <lambda>
        samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "L:\WebUI\system\python\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "L:\WebUI\webui\repositories\k-diffusion\k_diffusion\sampling.py", line 267, in sample_lms
        denoised = model(x, sigmas[i] * s_in, **extra_args)
      File "L:\WebUI\system\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "L:\WebUI\system\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
        return forward_call(*args, **kwargs)
      File "L:\WebUI\webui\modules\sd_samplers_cfg_denoiser.py", line 237, in forward
        x_out = self.inner_model(x_in, sigma_in, cond=make_condition_dict(cond_in, image_cond_in))
      File "L:\WebUI\system\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "L:\WebUI\system\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
        return forward_call(*args, **kwargs)
      File "L:\WebUI\webui\repositories\k-diffusion\k_diffusion\external.py", line 112, in forward
        eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
      File "L:\WebUI\webui\repositories\k-diffusion\k_diffusion\external.py", line 138, in get_eps
        return self.inner_model.apply_model(*args, **kwargs)
      File "L:\WebUI\webui\modules\sd_hijack_utils.py", line 18, in <lambda>
        setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
      File "L:\WebUI\webui\modules\sd_hijack_utils.py", line 32, in __call__
        return self.__orig_func(*args, **kwargs)
      File "L:\WebUI\webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 858, in apply_model
        x_recon = self.model(x_noisy, t, **cond)
      File "L:\WebUI\system\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "L:\WebUI\system\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
        return forward_call(*args, **kwargs)
      File "L:\WebUI\webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 1335, in forward
        out = self.diffusion_model(x, t, context=cc)
      File "L:\WebUI\system\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "L:\WebUI\system\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
        return forward_call(*args, **kwargs)
      File "L:\WebUI\webui\modules\sd_unet.py", line 91, in UNetModel_forward
        return original_forward(self, x, timesteps, context, *args, **kwargs)
      File "L:\WebUI\webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 797, in forward
        h = module(h, emb, context)
      File "L:\WebUI\system\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "L:\WebUI\system\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
        return forward_call(*args, **kwargs)
      File "L:\WebUI\webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 84, in forward
        x = layer(x, context)
      File "L:\WebUI\system\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "L:\WebUI\system\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
        return forward_call(*args, **kwargs)
      File "L:\WebUI\webui\repositories\stable-diffusion-stability-ai\ldm\modules\attention.py", line 334, in forward
        x = block(x, context=context[i])
      File "L:\WebUI\system\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "L:\WebUI\system\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
        return forward_call(*args, **kwargs)
      File "L:\WebUI\webui\repositories\stable-diffusion-stability-ai\ldm\modules\attention.py", line 269, in forward
        return checkpoint(self._forward, (x, context), self.parameters(), self.checkpoint)
      File "L:\WebUI\webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\util.py", line 121, in checkpoint
        return CheckpointFunction.apply(func, len(inputs), *args)
      File "L:\WebUI\system\python\lib\site-packages\torch\autograd\function.py", line 539, in apply
        return super().apply(*args, **kwargs)  # type: ignore[misc]
      File "L:\WebUI\webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\util.py", line 136, in forward
        output_tensors = ctx.run_function(*ctx.input_tensors)
      File "L:\WebUI\webui\repositories\stable-diffusion-stability-ai\ldm\modules\attention.py", line 273, in _forward
        x = self.attn2(self.norm2(x), context=context) + x
      File "L:\WebUI\system\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
        return self._call_impl(*args, **kwargs)
      File "L:\WebUI\system\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
        return forward_call(*args, **kwargs)
      File "L:\WebUI\webui\extensions\sd-webui-regional-prompter\scripts\attention.py", line 432, in forward
        opx = promptsepcalc(px, conp, mask, True, 2)
      File "L:\WebUI\webui\extensions\sd-webui-regional-prompter\scripts\attention.py", line 358, in promptsepcalc
        out = main_forward(module, x, context, mask, divide, self.isvanilla, userpp = userpp, width = dsw, height = dsh,      File "L:\WebUI\webui\extensions\sd-webui-regional-prompter\scripts\attention.py", line 64, in main_forward
        if inhr and not hiresfinished: hiresscaler(height,width,attn)
    TypeError: hiresscaler() missing 1 required positional argument: 'head'

Environment Web-UI version: 1.8.0 SD Version: 1.5 LoRA/LoCon/LoHa: none

Other Enabled Extensions

DWPose
OneButtonPrompt
adetailer
depthmap2mask
extension-style-vars
multidiffusion-upscaler-for-automatic1111
sd-dynamic-prompts
sd-webui-controlnet
sd-webui-hires-fix-tweaks
sd-webui-loractl
sd-webui-model-mixer
sd-webui-regional-prompter
stable-diffusion-webui-promptgen
asakleiman commented 6 months ago

I've been having this same problem for months. I thought something had become corrupted on my installation, so when I got around to making a fresh installation today and it persisted found this thread. The only extension I have in common is hires-fix-tweaks, and disabling that didn't do anything.

Currently - version: [v1.9.3] •  python: 3.10.6  •  torch: 2.1.2+cu121  •  xformers: 0.0.23.post1  •  gradio: 3.41.2  •  checkpoint: [463ebee3ad]

asakleiman commented 4 months ago

That fixes it, thank you!