search

v0xie / sd-webui-semantic-guidance

Unofficial implementation of "SEGA: Instructing Text-to-Image Models using Semantic Guidance". Semantic Guidance gives you more control over the semantics of an image given an additional text prompt. An extension for Automatic1111 WebUI.
GNU General Public License v3.0
65 stars 2 forks source link

semantic-guidance seems not work properly #23

Open A-B-01 opened 5 months ago

A-B-01 commented 5 months ago

Hi, v0xie, when I run your extension it raise an error.

Note, I made a test using in 648x864 px, because in 512px I noticed it seems work normally...

Is here:

** Error executing callback cfg_denoiser_callback for D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py Traceback (most recent call last): File "D:\stable-diffusion-webui\modules\script_callbacks.py", line 216, in cfg_denoiser_callback c.callback(params) File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 180, in y = lambda params: self.on_cfg_denoiser_callback(params, concept_conds, concepts_sega_params) File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 249, in on_cfg_denoiser_callback self.sega_routine_batch(params, batch_conds_list, batch_tensor, sega_params, text_cond, text_uncond) File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 336, in sega_routine_batch params.text_cond = params.text_cond + dir[i] RuntimeError: The size of tensor a (77) must match the size of tensor b (154) at non-singleton dimension 1

17%|██████████████▏ | 6/35 [00:29<02:29, 5.16s/it]*** Error executing callback cfg_denoiser_callback for D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py Traceback (most recent call last): File "D:\stable-diffusion-webui\modules\script_callbacks.py", line 216, in cfg_denoiser_callback c.callback(params) File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 180, in y = lambda params: self.on_cfg_denoiser_callback(params, concept_conds, concepts_sega_params) File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 249, in on_cfg_denoiser_callback self.sega_routine_batch(params, batch_conds_list, batch_tensor, sega_params, text_cond, text_uncond) File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 336, in sega_routine_batch params.text_cond = params.text_cond + dir[i] RuntimeError: The size of tensor a (77) must match the size of tensor b (154) at non-singleton dimension 1

                                                                                                                   *** Error executing callback cfg_denoiser_callback for D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py
Traceback (most recent call last):
  File "D:\stable-diffusion-webui\modules\script_callbacks.py", line 216, in cfg_denoiser_callback
    c.callback(params)
  File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 180, in <lambda>
    y = lambda params: self.on_cfg_denoiser_callback(params, concept_conds, concepts_sega_params)
  File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 249, in on_cfg_denoiser_callback
    self.sega_routine_batch(params, batch_conds_list, batch_tensor, sega_params, text_cond, text_uncond)
  File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 336, in sega_routine_batch
    params.text_cond = params.text_cond + dir[i]
RuntimeError: The size of tensor a (77) must match the size of tensor b (154) at non-singleton dimension 1

20%|████████████████▌ | 7/35 [00:33<02:12, 4.74s/it]*** Error executing callback cfg_denoiser_callback for D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py Traceback (most recent call last): File "D:\stable-diffusion-webui\modules\script_callbacks.py", line 216, in cfg_denoiser_callback c.callback(params) File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 180, in y = lambda params: self.on_cfg_denoiser_callback(params, concept_conds, concepts_sega_params) File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 249, in on_cfg_denoiser_callback self.sega_routine_batch(params, batch_conds_list, batch_tensor, sega_params, text_cond, text_uncond) File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 336, in sega_routine_batch params.text_cond = params.text_cond + dir[i] RuntimeError: The size of tensor a (77) must match the size of tensor b (154) at non-singleton dimension 1

                                                                                                                   *** Error executing callback cfg_denoiser_callback for D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py
Traceback (most recent call last):
  File "D:\stable-diffusion-webui\modules\script_callbacks.py", line 216, in cfg_denoiser_callback
    c.callback(params)
  File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 180, in <lambda>
    y = lambda params: self.on_cfg_denoiser_callback(params, concept_conds, concepts_sega_params)
  File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 249, in on_cfg_denoiser_callback
    self.sega_routine_batch(params, batch_conds_list, batch_tensor, sega_params, text_cond, text_uncond)
  File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 336, in sega_routine_batch
    params.text_cond = params.text_cond + dir[i]
RuntimeError: The size of tensor a (77) must match the size of tensor b (154) at non-singleton dimension 1

23%|██████████████████▉ | 8/35 [00:37<02:00, 4.47s/it]*** Error executing callback cfg_denoiser_callback for D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py Traceback (most recent call last): File "D:\stable-diffusion-webui\modules\script_callbacks.py", line 216, in cfg_denoiser_callback c.callback(params) File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 180, in y = lambda params: self.on_cfg_denoiser_callback(params, concept_conds, concepts_sega_params) File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 249, in on_cfg_denoiser_callback self.sega_routine_batch(params, batch_conds_list, batch_tensor, sega_params, text_cond, text_uncond) File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 336, in sega_routine_batch params.text_cond = params.text_cond + dir[i] RuntimeError: The size of tensor a (77) must match the size of tensor b (154) at non-singleton dimension 1

                                                                                                                   *** Error executing callback cfg_denoiser_callback for D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py
Traceback (most recent call last):
  File "D:\stable-diffusion-webui\modules\script_callbacks.py", line 216, in cfg_denoiser_callback
    c.callback(params)
  File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 180, in <lambda>
    y = lambda params: self.on_cfg_denoiser_callback(params, concept_conds, concepts_sega_params)
  File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 249, in on_cfg_denoiser_callback
    self.sega_routine_batch(params, batch_conds_list, batch_tensor, sega_params, text_cond, text_uncond)
  File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 336, in sega_routine_batch
    params.text_cond = params.text_cond + dir[i]
RuntimeError: The size of tensor a (77) must match the size of tensor b (154) at non-singleton dimension 1

97%|███████████████████████████████████████████████████████████████████████████████▋ | 34/35 [02:20<00:04, 4.01s/it]*** Error executing callback cfg_denoiser_callback for D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py Traceback (most recent call last): File "D:\stable-diffusion-webui\modules\script_callbacks.py", line 216, in cfg_denoiser_callback c.callback(params) File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 180, in y = lambda params: self.on_cfg_denoiser_callback(params, concept_conds, concepts_sega_params) File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 249, in on_cfg_denoiser_callback self.sega_routine_batch(params, batch_conds_list, batch_tensor, sega_params, text_cond, text_uncond) File "D:\stable-diffusion-webui\extensions\sd-webui-semantic-guidance\scripts\sega.py", line 336, in sega_routine_batch params.text_cond = params.text_cond + dir[i] RuntimeError: The size of tensor a (77) must match the size of tensor b (154) at non-singleton dimension 1

100%|██████████████████████████████████████████████████████████████████████████████████| 35/35 [02:22<00:00, 4.07s/it] Total progress: 100%|██████████████████████████████████████████████████████████████████| 35/35 [02:21<00:00, 4.04s/it] Total progress: 100%|██████████████████████████████████████████████████████████████████| 35/35 [02:21<00:00, 4.01s/it]

RomanKuschanow commented 3 months ago

I have the same problem, but I noticed that this extension works properly with another extension. Long story short, I changed lines 332-336 in sega.py form this:

if sampling_step >= warmup_period:
        if isinstance(params.text_cond, dict):
                params.text_cond[key] = params.text_cond[key] + dir[i]
        else:
                params.text_cond = params.text_cond + dir[i]

to this:

if sampling_step >= warmup_period:
        if isinstance(params.text_cond, dict):
                params.text_cond[key] = text_cond[key] + dir[i]
        else:
                params.text_cond = text_cond[key] + dir[i]

and it starts working properly but it slightly changes the seeds:

first without semantic guidance, second – with default params (except warmup, I changed it from 10 to 0), semantic guidance prompt: "monochrome"

image image

zethfoxster commented 2 days ago

it was working fine for a week, didnt change anything and suddenly i too am getting this error spam... i found out it was "Influencer-negatives-by-stable-yogi-neg" a negitive embed i was trying out. also found there are very common negitive embeds that are very popular that trigger this such as epiCNegative