search

lshqqytiger / stable-diffusion-webui-amdgpu

Stable Diffusion web UI
GNU Affero General Public License v3.0
1.69k stars 178 forks source link

[Bug]: Cross attention optimization not switch, only InvokeAI (it's slow) // How switch Cross attention optimization ? #256

Closed TimmekHW closed 10 months ago

TimmekHW commented 10 months ago

Is there an existing issue for this?

What happened?

I can't change Cross attention optimization. In 1111 and the DirectML branch, without Olive, "spd-mem" is the fastest. And with --onnx or --olive I can't change this value.

Steps to reproduce the problem

  1. Go to --olive --backend directml
  2. Press Setting > Optimizations > Cross attention optimization
  3. Nothing

What should have happened?

I can't change Cross attention optimization. In 1111 and the DirectML branch, without Olive, "spd-mem" is the fastest. And with --onnx or --olive I can't change this value.

Version or Commit where the problem happens

Version: 1.5.2

What Python version are you running on ?

Python 3.10.x

What platforms do you use to access the UI ?

Windows

What device are you running WebUI on?

NVIDIA GPUs, AMD GPUs

Cross attention optimization

Automatic

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

--olive --backend directml --autolaunch --listen --port 4756 --skip-torch-cuda-test --api --disable-safe-unpickle
git pull

List of extensions

No

Console logs

Model hassakumodel loaded.
Applying attention optimization: InvokeAI... done.
Running on local URL:  http://0.0.0.0:4756

To create a public link, set `share=True` in `launch()`.
Startup time: 21.0s (launcher: 6.4s, import torch: 3.2s, import gradio: 1.6s, setup paths: 1.9s, other imports: 2.5s, load scripts: 0.7s, create ui: 0.3s, gradio launch: 4.3s).
changing setting cross_attention_optimization to Doggettx: AttributeError
Traceback (most recent call last):
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\shared.py", line 647, in set
    self.data_labels[key].onchange()
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\call_queue.py", line 14, in f
    res = func(*args, **kwargs)
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\webui.py", line 243, in <lambda>
    shared.opts.onchange("cross_attention_optimization", wrap_queued_call(lambda: modules.sd_hijack.model_hijack.redo_hijack(shared.sd_model)), call=False)
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\sd_hijack.py", line 290, in redo_hijack
    self.undo_hijack(m)
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\sd_hijack.py", line 266, in undo_hijack
    ldm.modules.diffusionmodules.openaimodel.UNetModel.forward = ldm.modules.diffusionmodules.openaimodel.copy_of_UNetModel_forward_for_webui
AttributeError: module 'ldm.modules.diffusionmodules.openaimodel' has no attribute 'copy_of_UNetModel_forward_for_webui'
changing setting cross_attention_optimization to sub-quadratic: AttributeError
Traceback (most recent call last):
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\shared.py", line 647, in set
    self.data_labels[key].onchange()
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\call_queue.py", line 14, in f
    res = func(*args, **kwargs)
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\webui.py", line 243, in <lambda>
    shared.opts.onchange("cross_attention_optimization", wrap_queued_call(lambda: modules.sd_hijack.model_hijack.redo_hijack(shared.sd_model)), call=False)
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\sd_hijack.py", line 290, in redo_hijack
    self.undo_hijack(m)
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\sd_hijack.py", line 266, in undo_hijack
    ldm.modules.diffusionmodules.openaimodel.UNetModel.forward = ldm.modules.diffusionmodules.openaimodel.copy_of_UNetModel_forward_for_webui
AttributeError: module 'ldm.modules.diffusionmodules.openaimodel' has no attribute 'copy_of_UNetModel_forward_for_webui'
changing setting cross_attention_optimization to None: AttributeError
Traceback (most recent call last):
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\shared.py", line 647, in set
    self.data_labels[key].onchange()
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\call_queue.py", line 14, in f
    res = func(*args, **kwargs)
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\webui.py", line 243, in <lambda>
    shared.opts.onchange("cross_attention_optimization", wrap_queued_call(lambda: modules.sd_hijack.model_hijack.redo_hijack(shared.sd_model)), call=False)
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\sd_hijack.py", line 290, in redo_hijack
    self.undo_hijack(m)
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\sd_hijack.py", line 266, in undo_hijack
    ldm.modules.diffusionmodules.openaimodel.UNetModel.forward = ldm.modules.diffusionmodules.openaimodel.copy_of_UNetModel_forward_for_webui
AttributeError: module 'ldm.modules.diffusionmodules.openaimodel' has no attribute 'copy_of_UNetModel_forward_for_webui'
changing setting cross_attention_optimization to InvokeAI: AttributeError
Traceback (most recent call last):
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\shared.py", line 647, in set
    self.data_labels[key].onchange()
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\call_queue.py", line 14, in f
    res = func(*args, **kwargs)
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\webui.py", line 243, in <lambda>
    shared.opts.onchange("cross_attention_optimization", wrap_queued_call(lambda: modules.sd_hijack.model_hijack.redo_hijack(shared.sd_model)), call=False)
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\sd_hijack.py", line 290, in redo_hijack
    self.undo_hijack(m)
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\sd_hijack.py", line 266, in undo_hijack
    ldm.modules.diffusionmodules.openaimodel.UNetModel.forward = ldm.modules.diffusionmodules.openaimodel.copy_of_UNetModel_forward_for_webui
AttributeError: module 'ldm.modules.diffusionmodules.openaimodel' has no attribute 'copy_of_UNetModel_forward_for_webui'
changing setting cross_attention_optimization to Doggettx: AttributeError
Traceback (most recent call last):
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\shared.py", line 647, in set
    self.data_labels[key].onchange()
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\call_queue.py", line 14, in f
    res = func(*args, **kwargs)
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\webui.py", line 243, in <lambda>
    shared.opts.onchange("cross_attention_optimization", wrap_queued_call(lambda: modules.sd_hijack.model_hijack.redo_hijack(shared.sd_model)), call=False)
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\sd_hijack.py", line 290, in redo_hijack
    self.undo_hijack(m)
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\sd_hijack.py", line 266, in undo_hijack
    ldm.modules.diffusionmodules.openaimodel.UNetModel.forward = ldm.modules.diffusionmodules.openaimodel.copy_of_UNetModel_forward_for_webui
AttributeError: module 'ldm.modules.diffusionmodules.openaimodel' has no attribute 'copy_of_UNetModel_forward_for_webui'
changing setting cross_attention_optimization to sub-quadratic: AttributeError
Traceback (most recent call last):
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\shared.py", line 647, in set
    self.data_labels[key].onchange()
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\call_queue.py", line 14, in f
    res = func(*args, **kwargs)
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\webui.py", line 243, in <lambda>
    shared.opts.onchange("cross_attention_optimization", wrap_queued_call(lambda: modules.sd_hijack.model_hijack.redo_hijack(shared.sd_model)), call=False)
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\sd_hijack.py", line 290, in redo_hijack
    self.undo_hijack(m)
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\sd_hijack.py", line 266, in undo_hijack
    ldm.modules.diffusionmodules.openaimodel.UNetModel.forward = ldm.modules.diffusionmodules.openaimodel.copy_of_UNetModel_forward_for_webui
AttributeError: module 'ldm.modules.diffusionmodules.openaimodel' has no attribute 'copy_of_UNetModel_forward_for_webui'
changing setting cross_attention_optimization to V1 - original v1: AttributeError
Traceback (most recent call last):
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\shared.py", line 647, in set
    self.data_labels[key].onchange()
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\call_queue.py", line 14, in f
    res = func(*args, **kwargs)
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\webui.py", line 243, in <lambda>
    shared.opts.onchange("cross_attention_optimization", wrap_queued_call(lambda: modules.sd_hijack.model_hijack.redo_hijack(shared.sd_model)), call=False)
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\sd_hijack.py", line 290, in redo_hijack
    self.undo_hijack(m)
  File "C:\Users\Timmek\olive\examples\directml\stable_diffusion\stable-diffusion-webui-directml\modules\sd_hijack.py", line 266, in undo_hijack
    ldm.modules.diffusionmodules.openaimodel.UNetModel.forward = ldm.modules.diffusionmodules.openaimodel.copy_of_UNetModel_forward_for_webui
AttributeError: module 'ldm.modules.diffusionmodules.openaimodel' has no attribute 'copy_of_UNetModel_forward_for_webui'

Additional information

No response

TimmekHW commented 10 months ago

venv folder deleted several times. did not help (

I also changed it through ui-config, I still use InvokeAI (slow)

lshqqytiger commented 10 months ago

All existing optimization methods are meaningless with --olive. The log will say you are using InvokeAI, but you are actually using the default attention processor of OnnxStableDiffusionPipeline. I will add a attention processor option in settings.