Closed xdomiall closed 1 year ago
Symbiomatrix
commented
1 year ago Hello. Are you getting any errors? On #131 we discovered there was some change on the dev branch, which apparently is supposed to allow varying sampler & prompts mid run optionally, but didn't use all the standard calls and broke the extension. So it could be related, Haven't gotten confirmation and I'm definitely seeing no features I care to upgrade for in the latest version.
xdomiall
commented
1 year ago Hi @Symbiomatrix! Thank you for looking into this, I really appreciate it! Unfortunately I receive no error messages. I've checked the thread that you've linked and it does make a lot of sense if this is related to that change. Seeing as the main branch got updated twice in the last weeks hopefully this will be addressed there soonish
Symbiomatrix
commented
1 year ago @hako-mikan Would you like to have a look at this? Not much to go on without access to the recent version.
hako-mikan
commented
1 year ago @Symbiomatrix OK. I will have a look it.
hako-mikan
commented
1 year ago Fixed. but its was quick fix so prompt for hires fix is now disabled.
xdomiall
commented
1 year ago Thank you so much @hako-mikan! I really appreciate your work and your quick response! I've tested and everything works perfectly now! Would love to show my support, do you have a Patreon or Ko-Fi or anything similar?
anae-git
commented
1 year ago I'm still having trouble with this breaking when the hires fix kicks in. Output is:
u_count 196 u_count '%' divide 0.82it/s]
0 mycat1-000004added_by_regional_prompter1007 diffusion_model_output_blocks_9_1_transformer_blocks_0_attn1_to_q <main.py.LoraModule object at 0x7f1f6abf0fd0> 0 mich5added_by_regional_prompter1008 diffusion_model_output_blocks_9_1_transformer_blocks_0_attn1_to_q <main.py.LoraModule object at 0x7f1f7b038280>
u_count 197 u_count '%' divide 1 0.9 mycat1-000004added_by_regional_prompter1007 diffusion_model_output_blocks_9_1_transformer_blocks_0_attn1_to_q <main.py.LoraModule object at 0x7f1f6abf0fd0>
0 mich5added_by_regional_prompter1008 diffusion_model_output_blocks_9_1_transformer_blocks_0_attn1_to_q <main.py.LoraModule object at 0x7f1f7b038280>
u_count 198 u_count '%' divide 2 0 mycat1-000004added_by_regional_prompter1007 diffusion_model_output_blocks_9_1_transformer_blocks_0_attn1_to_q <main.py.LoraModule object at 0x7f1f6abf0fd0>
0.9 mich5added_by_regional_prompter1008 diffusion_model_output_blocks_9_1_transformer_blocks_0_attn1_to_q <main.py.LoraModule object at 0x7f1f7b038280> u_count 199 u_count '%' divide 3
0.0 mycat1-000004added_by_regional_prompter1007 diffusion_model_output_blocks_9_1_transformer_blocks_0_attn1_to_q <main.py.LoraModule object at 0x7f1f6abf0fd0>
0.0 mich5added_by_regional_prompter1008 diffusion_model_output_blocks_9_1_transformer_blocks_0_attn1_to_q <main.py.LoraModule object at 0x7f1f7b038280> filterlength : 3
tensor(5.7322, device='cuda:0') tensor(5.7299, device='cuda:0')
tensor(5.7316, device='cuda:0')
tensor(5.7317, device='cuda:0') x = torch.Size([4, 4, 80, 64])i = 0, cond = 0, uncon = 3
x = torch.Size([4, 4, 80, 64])i = 1, cond = 1, uncon = 3 x = torch.Size([4, 4, 80, 64])i = 2, cond = 2, uncon = 3
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:07<00:00, 6.75it/s]so far, so good. We finished the non-hires part and the mycat1-000004 and mich5 loras were applied correctly as per the preview.
But then:
locon load lora method[13:56, 6.72it/s]
locon load lora method
0%| | 0/30 [00:00<?, ?it/s]u_count 200 u_count '%' divide 0
0%| | 0/30 [00:00<?, ?it/s]
Error completing request
Arguments: ('task(c8gq7pa9ll83cmu)', 'A photo of a cat and a dog, BREAK\n((realistic)) BREAK\nmycat cat <lora:mycat1-000004:0.9>, on a chair BREAK\nMich dog <lora:mich5:0.9>, on the floor\n', 'realisticvision-negative-embedding', [], 50, 16, False, False, 1, 1, 6, 964681097.0, -1.0, 0, 0, 0, False, 640, 512, True, 0.25, 2, '4x-UltraSharp', 30, 0, 0, 0, '', '', [], 0, 0, 0, 0, 0, 0.25, False, False, {'ad_model': 'face_yolov8n.pt', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1}, {'ad_model': 'None', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1}, False, '', 0, False, False, 'LoRA', 'None', 0, 0, 'LoRA', 'None', 0, 0, 'LoRA', 'None', 0, 0, 'LoRA', 'None', 0, 0, 'LoRA', 'None', 0, 0, None, 'Refresh models', <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x7f1f7b2a22f0>, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x7f1f7b2a21d0>, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x7f1f7b2e6320>, 'NONE:0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0\nALL:1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1\nINS:1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0\nIND:1,0,0,0,1,1,1,0,0,0,0,0,0,0,0,0,0\nINALL:1,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,0\nMIDD:1,0,0,0,1,1,1,1,1,1,1,1,0,0,0,0,0\nOUTD:1,0,0,0,0,0,0,0,1,1,1,1,0,0,0,0,0\nOUTS:1,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1\nOUTALL:1,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1\nALL0.5:0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5', True, 0, 'values', '0,0.25,0.5,0.75,1', 'Block ID', 'IN05-OUT05', 'none', '', '0.5,1', 'BASE,IN00,IN01,IN02,IN03,IN04,IN05,IN06,IN07,IN08,IN09,IN10,IN11,M00,OUT00,OUT01,OUT02,OUT03,OUT04,OUT05,OUT06,OUT07,OUT08,OUT09,OUT10,OUT11', 1.0, 'black', '20', False, 'ATTNDEEPON:IN05-OUT05:attn:1\n\nATTNDEEPOFF:IN05-OUT05:attn:0\n\nPROJDEEPOFF:IN05-OUT05:proj:0\n\nXYZ:::1', False, True, True, 'Matrix', 'Horizontal', 'Mask', 'Prompt', '1,1', '0.2', True, True, False, 'Latent', True, '0', '0', '0.4', None, False, False, False, False, '1:1,1:2,1:2', '0:0,0:0,0:1', '0.2,0.8,0.8', 150, 0.2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, None, None, False, None, None, False, None, None, False, 50) {}
Traceback (most recent call last):
File "/home/sduser/sd-webui/modules/call_queue.py", line 57, in f
res = list(func(*args, **kwargs))
File "/home/sduser/sd-webui/modules/call_queue.py", line 37, in f
res = func(*args, **kwargs)
File "/home/sduser/sd-webui/modules/txt2img.py", line 57, in txt2img
processed = processing.process_images(p)
File "/home/sduser/sd-webui/modules/processing.py", line 610, in process_images
res = process_images_inner(p)
File "/home/sduser/sd-webui/extensions/sd-webui-controlnet/scripts/batch_hijack.py", line 42, in processing_process_images_hijack
return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
File "/home/sduser/sd-webui/modules/processing.py", line 728, in process_images_inner
samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
File "/home/sduser/sd-webui/modules/processing.py", line 1058, in sample
samples = self.sampler.sample_img2img(self, samples, noise, self.hr_c, self.hr_uc, steps=self.hr_second_pass_steps or self.steps, image_conditioning=image_conditioning)
File "/home/sduser/sd-webui/modules/sd_samplers_kdiffusion.py", line 356, in sample_img2img
samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
File "/home/sduser/sd-webui/modules/sd_samplers_kdiffusion.py", line 257, in launch_sampling
return func()
File "/home/sduser/sd-webui/modules/sd_samplers_kdiffusion.py", line 356, in <lambda>
samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
File "/home/sduser/automatic/venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/sduser/sd-webui/repositories/k-diffusion/k_diffusion/sampling.py", line 594, in sample_dpmpp_2m
denoised = model(x, sigmas[i] * s_in, **extra_args)
File "/home/sduser/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/sduser/sd-webui/modules/sd_samplers_kdiffusion.py", line 143, in forward
x_out[a:b] = self.inner_model(x_in[a:b], sigma_in[a:b], cond=make_condition_dict([cond_in[a:b]], image_cond_in[a:b]))
File "/home/sduser/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/sduser/sd-webui/repositories/k-diffusion/k_diffusion/external.py", line 112, in forward
eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
File "/home/sduser/sd-webui/repositories/k-diffusion/k_diffusion/external.py", line 138, in get_eps
return self.inner_model.apply_model(*args, **kwargs)
File "/home/sduser/sd-webui/modules/sd_hijack_utils.py", line 17, in <lambda>
setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
File "/home/sduser/sd-webui/modules/sd_hijack_utils.py", line 28, in __call__
return self.__orig_func(*args, **kwargs)
File "/home/sduser/sd-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 858, in apply_model
x_recon = self.model(x_noisy, t, **cond)
File "/home/sduser/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/sduser/sd-webui/repositories/stable-diffusion-stability-ai/ldm/models/diffusion/ddpm.py", line 1335, in forward
out = self.diffusion_model(x, t, context=cc)
File "/home/sduser/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/sduser/sd-webui/repositories/stable-diffusion-stability-ai/ldm/modules/diffusionmodules/openaimodel.py", line 789, in forward
emb = self.time_embed(t_emb)
File "/home/sduser/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/sduser/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/container.py", line 217, in forward
input = module(input)
File "/home/sduser/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/home/sduser/sd-webui/extensions/sd-webui-regional-prompter/scripts/latent.py", line 472, in lora_Linear_forward
return lora_forward(self, input, torch.nn.Linear_forward_before_lora(self, input))
File "/home/sduser/sd-webui/extensions/sd-webui-regional-prompter/scripts/latent.py", line 376, in lora_forward
regioner.u_start()
File "/home/sduser/sd-webui/extensions/sd-webui-regional-prompter/scripts/latent.py", line 353, in u_start
lora.loaded_loras[i].multiplier = self.mlist[lora.loaded_loras[i].name]
KeyError: 'mycat1-000004' 
ianfedev
commented
1 year ago I'm having the same error as you... Did you find any way to fix it?
Symbiomatrix
commented
1 year ago @xdomiall @ianfedev There are some notes about bugfixes to hires in webui V1.4 RC, how does it look?
AntitrustEnthusiast
commented
1 year ago I'm on 1.4.0-RC, and I'm seeing the same issue. Basically seems to destroy detail/add a bunch of noise in added regions.
Simple example with prompt apple ADDCOL banana
Minimal reproducible example:
No hi-res fix:
Hi-res fix:
Kl0ndike
commented
1 year ago I'm also having errors with hires fix on A1111 1.3.2. It seems to get the first region/top left section fairly okay and then disintegrates the rest of the image, for lack of a better term. Only having the issue when using hires fix to blow up an image after getting a good render without it.
hako-mikan
commented
1 year ago OK. I‘ll check it.
hako-mikan
commented
1 year ago fixed
TeutonJon78
commented
1 year ago After updating the extension that has this fix, now I'm getting this error every generation:
16:19:07-993442 ERROR Running script process: D:\Graphic
Design\AI\vladmandic-automatic\extensions\sd-webui-regional-prompter\scripts\rp.py:
AttributeError
╭───────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────╮
│ D:\Graphic Design\AI\vladmandic-automatic\modules\scripts.py:406 in process │
│ │
│ 405 │ │ │ │ args = p.per_script_args.get(script.title(), p.script_args[script.args_f │
│ ❱ 406 │ │ │ │ script.process(p, *args, **kwargs) │
│ 407 │ │ │ │ s.append(f'{script.title()}:{round(time.time()-t0, 2)}s') │
│ │
│ D:\Graphic Design\AI\vladmandic-automatic\extensions\sd-webui-regional-prompter\scripts\rp.py:450 in process │
│ │
│ 449 │ │ self = bratioprompt(self, bratios) │
│ ❱ 450 │ │ p = hrdealer(p) │
│ 451 │
│ │
│ D:\Graphic Design\AI\vladmandic-automatic\extensions\sd-webui-regional-prompter\scripts\rp.py:573 in hrdealer │
│ │
│ 572 def hrdealer(p): │
│ ❱ 573 │ p.hr_prompts = p.prompts │
│ 574 │ p.hr_negative_prompts = p.negative_prompts │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
AttributeError: 'StableDiffusionProcessingTxt2Img' object has no attribute 'prompts'I should note that I have hires.fix disabled.
hako-mikan
commented
1 year ago Seems to be a problem specific to the fork version. Maybe fixed. Test it.
theddr
commented
1 year ago Seems to be a problem specific to the fork version. Maybe fixed. Test it.
KeyError with lora in Latent mode hires.fix enabled. webui 1.3.2
but it seems good in webui 1.2.1
TeutonJon78
commented
1 year ago Updated, and error still there, but switched to different line. I should note before and now that it still does seem to generate the image correctly (I was reproducing an existing image), so not sure what the error does other than spam the console.
╭───────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────╮
│ D:\Graphic Design\AI\vladmandic-automatic\modules\scripts.py:430 in process_batch │
│ │
│ 429 │ │ │ │ args = p.per_script_args.get(script.title(), p.script_args[script.args_f │
│ ❱ 430 │ │ │ │ script.process_batch(p, *args, **kwargs) │
│ 431 │ │ │ │ s.append(f'{script.title()}:{round(time.time()-t0, 2)}s') │
│ │
│ D:\Graphic Design\AI\vladmandic-automatic\extensions\sd-webui-regional-prompter\scripts\rp.py:480 in process_batch │
│ │
│ 479 │ │ │ │ │ self.lora_applied = True │
│ ❱ 480 │ │ │ print(p.prompts) │
│ 481 │ │ │ print(p.hr_prompts) │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
AttributeError: 'StableDiffusionProcessingTxt2Img' object has no attribute 'prompts'Edit: I did check in auto1111 1.3.2 (directML port) and it didn't throw any errors there.
hako-mikan
commented
1 year ago Fixed for LoRA. In this fix additional LoRAs in hr_prompt are disabled.
TeutonJon78
commented
1 year ago Thanks, it's working without throwing errors now.
Hi everyone,
It seeems like since updating to a1111 1.3.0 the regional prompter no longer works when hires fix is applied. Hires applied:
Hires disabled:
This is on a clean install of 1.3.0 with no other extensions. This occurs regardless of the model and hires upscaler used. Any help would be really appreciated! Thank you!
Prompt:
2girls,anime, desert, full body ADDCOMM airplane ADDROW 2girls, red hair, military uniform,transparent pants,steel boots, sitting on the ground, bloody face, sad, laughing, claw_pose,weapon ADDCOL stairs ADDCOL 2girls, blue hair, cat ears, cat paws, fangs, crying, hooded robe. (happy),magic, voluptous, big tits, sports fit Negative prompt: EasyNegative, lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, (worst quality:1.2), low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry, lowres graffiti, (low quality lowres simple background:1.1), bad_prompt_version2 Steps: 25, Sampler: DPM++ 2M SDE Karras, CFG scale: 6, Seed: 3787083238, Size: 576x832, Model hash: a7e662921a, Model: SS_gumysakurav3, Clip skip: 2, RP Active: True, RP Divide mode: Matrix, RP Matrix submode: Horizontal, RP Mask submode: Mask, RP Prompt submode: Prompt, RP Calc Mode: Attention, RP Ratios: "0.2,0.4;1,2,1,2", RP Base Ratios: 0.2, RP Use Base: False, RP Use Common: True, RP Use Ncommon: False, RP Change AND: False, RP LoRA Neg Te Ratios: 0, RP LoRA Neg U Ratios: 0, RP threshold: 0.4