[Bug]: image generation will fail or will succeed with some tags but not with others. I could not found a consistent rule on what tags it does this

[dilectiogames]

Checklist

• [X] The issue exists after disabling all extensions
• [ ] The issue exists on a clean installation of webui
• [ ] The issue is caused by an extension, but I believe it is caused by a bug in the webui
• [X] The issue exists in the current version of the webui
• [X] The issue has not been reported before recently
• [ ] The issue has been reported before but has not been fixed yet

What happened?

image generation will fail or will succeed with some tags but not with others. I could not found a consistent rule on what tags it does this

Steps to reproduce the problem

i have no idea why some tags work and other don't

What should have happened?

generate image regardless of tags

What browsers do you use to access the UI ?

Mozilla Firefox

Sysinfo

        "traceback": [
            [
                "E:\\AI\\stable-diffusion-webui\\modules\\call_queue.py, line 74, f",
                "res = list(func(*args, **kwargs))"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\modules\\call_queue.py, line 53, f",
                "res = func(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\modules\\call_queue.py, line 37, f",
                "res = func(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\modules\\txt2img.py, line 109, txt2img",
                "processed = processing.process_images(p)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\modules\\processing.py, line 847, process_images",
                "res = process_images_inner(p)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\modules\\processing.py, line 988, process_images_inner",
                "samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\modules\\processing.py, line 1346, sample",
                "samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\modules\\sd_samplers_kdiffusion.py, line 230, sample",
                "samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\modules\\sd_samplers_common.py, line 272, launch_sampling",
                "return func()"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\modules\\sd_samplers_kdiffusion.py, line 230, <lambda>",
                "samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\utils\\_contextlib.py, line 115, decorate_context",
                "return func(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\repositories\\k-diffusion\\k_diffusion\\sampling.py, line 145, sample_euler_ancestral",
                "denoised = model(x, sigmas[i] * s_in, **extra_args)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1518, _wrapped_call_impl",
                "return self._call_impl(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1527, _call_impl",
                "return forward_call(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\modules\\sd_samplers_cfg_denoiser.py, line 249, forward",
                "x_out = self.inner_model(x_in, sigma_in, cond=make_condition_dict(cond_in, image_cond_in))"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1518, _wrapped_call_impl",
                "return self._call_impl(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1527, _call_impl",
                "return forward_call(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\repositories\\k-diffusion\\k_diffusion\\external.py, line 112, forward",
                "eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\repositories\\k-diffusion\\k_diffusion\\external.py, line 138, get_eps",
                "return self.inner_model.apply_model(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\modules\\sd_models_xl.py, line 43, apply_model",
                "return self.model(x, t, cond)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1518, _wrapped_call_impl",
                "return self._call_impl(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1568, _call_impl",
                "result = forward_call(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\modules\\sd_hijack_utils.py, line 22, <lambda>",
                "setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\modules\\sd_hijack_utils.py, line 34, __call__",
                "return self.__sub_func(self.__orig_func, *args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\modules\\sd_hijack_unet.py, line 50, apply_model",
                "result = orig_func(self, x_noisy.to(devices.dtype_unet), t.to(devices.dtype_unet), cond, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\repositories\\generative-models\\sgm\\modules\\diffusionmodules\\wrappers.py, line 28, forward",
                "return self.diffusion_model("
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1518, _wrapped_call_impl",
                "return self._call_impl(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1527, _call_impl",
                "return forward_call(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\modules\\sd_unet.py, line 91, UNetModel_forward",
                "return original_forward(self, x, timesteps, context, *args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\repositories\\generative-models\\sgm\\modules\\diffusionmodules\\openaimodel.py, line 998, forward",
                "h = module(h, emb, context)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1518, _wrapped_call_impl",
                "return self._call_impl(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1527, _call_impl",
                "return forward_call(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\repositories\\generative-models\\sgm\\modules\\diffusionmodules\\openaimodel.py, line 100, forward",
                "x = layer(x, context)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1518, _wrapped_call_impl",
                "return self._call_impl(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1527, _call_impl",
                "return forward_call(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\repositories\\generative-models\\sgm\\modules\\attention.py, line 627, forward",
                "x = block(x, context=context[i])"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1518, _wrapped_call_impl",
                "return self._call_impl(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1527, _call_impl",
                "return forward_call(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\repositories\\generative-models\\sgm\\modules\\attention.py, line 459, forward",
                "return checkpoint("
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\repositories\\generative-models\\sgm\\modules\\diffusionmodules\\util.py, line 167, checkpoint",
                "return func(*inputs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\repositories\\generative-models\\sgm\\modules\\attention.py, line 483, _forward",
                "x = self.ff(self.norm3(x)) + x"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1518, _wrapped_call_impl",
                "return self._call_impl(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1527, _call_impl",
                "return forward_call(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\repositories\\generative-models\\sgm\\modules\\attention.py, line 108, forward",
                "return self.net(x)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1518, _wrapped_call_impl",
                "return self._call_impl(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1527, _call_impl",
                "return forward_call(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\container.py, line 215, forward",
                "input = module(input)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1518, _wrapped_call_impl",
                "return self._call_impl(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1527, _call_impl",
                "return forward_call(*args, **kwargs)"
            ],
            [
                "E:\\AI\\stable-diffusion-webui\\repositories\\generative-models\\sgm\\modules\\attention.py, line 89, forward",
                "return x * F.gelu(gate)"
            ]
        ]
    }
],

Console logs

"traceback": [
                [
                    "E:\\AI\\stable-diffusion-webui\\modules\\call_queue.py, line 74, f",
                    "res = list(func(*args, **kwargs))"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\modules\\call_queue.py, line 53, f",
                    "res = func(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\modules\\call_queue.py, line 37, f",
                    "res = func(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\modules\\txt2img.py, line 109, txt2img",
                    "processed = processing.process_images(p)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\modules\\processing.py, line 847, process_images",
                    "res = process_images_inner(p)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\modules\\processing.py, line 988, process_images_inner",
                    "samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\modules\\processing.py, line 1346, sample",
                    "samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\modules\\sd_samplers_kdiffusion.py, line 230, sample",
                    "samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\modules\\sd_samplers_common.py, line 272, launch_sampling",
                    "return func()"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\modules\\sd_samplers_kdiffusion.py, line 230, <lambda>",
                    "samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\utils\\_contextlib.py, line 115, decorate_context",
                    "return func(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\repositories\\k-diffusion\\k_diffusion\\sampling.py, line 145, sample_euler_ancestral",
                    "denoised = model(x, sigmas[i] * s_in, **extra_args)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1518, _wrapped_call_impl",
                    "return self._call_impl(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1527, _call_impl",
                    "return forward_call(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\modules\\sd_samplers_cfg_denoiser.py, line 249, forward",
                    "x_out = self.inner_model(x_in, sigma_in, cond=make_condition_dict(cond_in, image_cond_in))"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1518, _wrapped_call_impl",
                    "return self._call_impl(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1527, _call_impl",
                    "return forward_call(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\repositories\\k-diffusion\\k_diffusion\\external.py, line 112, forward",
                    "eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\repositories\\k-diffusion\\k_diffusion\\external.py, line 138, get_eps",
                    "return self.inner_model.apply_model(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\modules\\sd_models_xl.py, line 43, apply_model",
                    "return self.model(x, t, cond)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1518, _wrapped_call_impl",
                    "return self._call_impl(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1568, _call_impl",
                    "result = forward_call(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\modules\\sd_hijack_utils.py, line 22, <lambda>",
                    "setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\modules\\sd_hijack_utils.py, line 34, __call__",
                    "return self.__sub_func(self.__orig_func, *args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\modules\\sd_hijack_unet.py, line 50, apply_model",
                    "result = orig_func(self, x_noisy.to(devices.dtype_unet), t.to(devices.dtype_unet), cond, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\repositories\\generative-models\\sgm\\modules\\diffusionmodules\\wrappers.py, line 28, forward",
                    "return self.diffusion_model("
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1518, _wrapped_call_impl",
                    "return self._call_impl(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1527, _call_impl",
                    "return forward_call(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\modules\\sd_unet.py, line 91, UNetModel_forward",
                    "return original_forward(self, x, timesteps, context, *args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\repositories\\generative-models\\sgm\\modules\\diffusionmodules\\openaimodel.py, line 998, forward",
                    "h = module(h, emb, context)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1518, _wrapped_call_impl",
                    "return self._call_impl(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1527, _call_impl",
                    "return forward_call(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\repositories\\generative-models\\sgm\\modules\\diffusionmodules\\openaimodel.py, line 100, forward",
                    "x = layer(x, context)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1518, _wrapped_call_impl",
                    "return self._call_impl(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1527, _call_impl",
                    "return forward_call(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\repositories\\generative-models\\sgm\\modules\\attention.py, line 627, forward",
                    "x = block(x, context=context[i])"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1518, _wrapped_call_impl",
                    "return self._call_impl(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1527, _call_impl",
                    "return forward_call(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\repositories\\generative-models\\sgm\\modules\\attention.py, line 459, forward",
                    "return checkpoint("
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\repositories\\generative-models\\sgm\\modules\\diffusionmodules\\util.py, line 167, checkpoint",
                    "return func(*inputs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\repositories\\generative-models\\sgm\\modules\\attention.py, line 483, _forward",
                    "x = self.ff(self.norm3(x)) + x"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1518, _wrapped_call_impl",
                    "return self._call_impl(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1527, _call_impl",
                    "return forward_call(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\repositories\\generative-models\\sgm\\modules\\attention.py, line 108, forward",
                    "return self.net(x)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1518, _wrapped_call_impl",
                    "return self._call_impl(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1527, _call_impl",
                    "return forward_call(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\container.py, line 215, forward",
                    "input = module(input)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1518, _wrapped_call_impl",
                    "return self._call_impl(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\venv\\lib\\site-packages\\torch\\nn\\modules\\module.py, line 1527, _call_impl",
                    "return forward_call(*args, **kwargs)"
                ],
                [
                    "E:\\AI\\stable-diffusion-webui\\repositories\\generative-models\\sgm\\modules\\attention.py, line 89, forward",
                    "return x * F.gelu(gate)"
                ]
            ]
        }
    ],

Additional information

No response

[dilectiogames]

I believe there is some error in the calculation of tokens internally, when combining the prompt, it doesn't matter if there are more or less tokens. it can fail with 2 tokens or 200 tokens. just adding a new token will trigger the fail, or changing the weight,

this is SDXL Pony generation, forgot to add this detail. Basically it just gets crazy with the CUDA errors because of this. It doesn't make any sense.

At first I thought that fewer tags means less memory requirements but I can add 300 at the positive and 300 at the negative and it will work, remove one word, fails, remove some other random words, it works.

I'm not using any special tokens or notation, just standard tokens, no loras whatsoever.

i have all the extensions disabled