[Bug]: ONNX model conversion error when generating image

waylaa commented 4 months ago

Checklist

[X] The issue exists after disabling all extensions
[X] The issue exists on a clean installation of webui
[ ] The issue is caused by an extension, but I believe it is caused by a bug in the webui
[X] The issue exists in the current version of the webui
[X] The issue has not been reported before recently
[ ] The issue has been reported before but has not been fixed yet

What happened?

When trying to generate an image (with in this case, this model). The following error is being shown in the console.

============== Diagnostic Run torch.onnx.export version 2.0.0+cpu ==============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 1 ERROR ========================
ERROR: missing-standard-symbolic-function
=========================================
Exporting the operator 'aten::scaled_dot_product_attention' to ONNX opset version 14 is not supported. Please feel free to request support or submit a pull request on PyTorch GitHub: https://github.com/pytorch/pytorch/issues.
None
<Set verbose=True to see more details>

ONNX: Failed to convert model: model='corneos7thHeavenMix_v2.safetensors', error=Exporting the operator 'aten::scaled_dot_product_attention' to ONNX opset version 14 is not supported. Please feel free to request support or submit a pull request on PyTorch GitHub: https://github.com/pytorch/pytorch/issues.
ONNX: processing=StableDiffusionProcessingTxt2Img, pipeline=NoneType

Steps to reproduce the problem

Install this repo
Use this model (It's the only one I have tested)
Change these settings:
- OnnxRuntime ->☑️ 'Use ONNX Runtime instead of PyTorch implementation'
- OnnxRuntime -> ☑️ 'Enable Olive'
- OnnxRuntime -> ☑️ 'Olive models to process' (Text Encoder, Model, VAE)

What should have happened?

The model should have been converted successfully and then generate an image based on the given positive/negative prompt.

What browsers do you use to access the UI ?

Mozilla Firefox

Sysinfo

sysinfo-2024-02-09-20-47.json

Console logs

Already up to date.
venv "D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\venv\Scripts\Python.exe"
fatal: No names found, cannot describe anything.
Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
Version: 1.7.0
Commit hash: f2f190b73e7deb97a59386fc52d155e858355e42
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\pytorch_lightning\utilities\distributed.py:258: LightningDeprecationWarning: `pytorch_lightning.utilities.distributed.rank_zero_only` has been deprecated in v1.8.1 and will be removed in v2.0.0. You can import it from `pytorch_lightning.utilities` instead.
  rank_zero_deprecation(
Launching Web UI with arguments: --skip-torch-cuda-test --use-directml
Style database not found: D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\styles.csv
ONNX: selected=DmlExecutionProvider, available=['DmlExecutionProvider', 'CPUExecutionProvider']
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 2.4s (prepare environment: 4.0s, initialize shared: 1.0s, load scripts: 0.7s, create ui: 0.4s, gradio launch: 0.1s).
You have disabled the safety checker for <class 'diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion.StableDiffusionPipeline'> by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
Applying attention optimization: InvokeAI... done.
D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\modules\dml\hijack\transformers.py:13: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  min = torch.tensor(torch.finfo(dtype).min, device="cpu")
D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\transformers\models\clip\modeling_clip.py:284: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attn_weights.size() != (bsz * self.num_heads, tgt_len, src_len):
D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\transformers\models\clip\modeling_clip.py:292: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if causal_attention_mask.size() != (bsz, 1, tgt_len, src_len):
D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\transformers\models\clip\modeling_clip.py:324: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if attn_output.size() != (bsz * self.num_heads, tgt_len, self.head_dim):
============== Diagnostic Run torch.onnx.export version 2.0.0+cpu ==============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

ONNX: Successfully exported converted model: submodel=text_encoder
D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\diffusers\models\unets\unet_2d_condition.py:924: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if dim % default_overall_up_factor != 0:
D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\diffusers\models\downsampling.py:135: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert hidden_states.shape[1] == self.channels
D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\diffusers\models\downsampling.py:144: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert hidden_states.shape[1] == self.channels
D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\diffusers\models\upsampling.py:149: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert hidden_states.shape[1] == self.channels
D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\diffusers\models\upsampling.py:165: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if hidden_states.shape[0] >= 64:
D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\diffusers\models\unets\unet_2d_condition.py:1215: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if not return_dict:
============== Diagnostic Run torch.onnx.export version 2.0.0+cpu ==============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 1 ERROR ========================
ERROR: missing-standard-symbolic-function
=========================================
Exporting the operator 'aten::scaled_dot_product_attention' to ONNX opset version 14 is not supported. Please feel free to request support or submit a pull request on PyTorch GitHub: https://github.com/pytorch/pytorch/issues.
None
<Set verbose=True to see more details>

ONNX: Failed to convert model: model='corneos7thHeavenMix_v2.safetensors', error=Exporting the operator 'aten::scaled_dot_product_attention' to ONNX opset version 14 is not supported. Please feel free to request support or submit a pull request on PyTorch GitHub: https://github.com/pytorch/pytorch/issues.
You have disabled the safety checker for <class 'diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion.StableDiffusionPipeline'> by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
ONNX: processing=StableDiffusionProcessingTxt2Img, pipeline=OnnxRawPipeline
*** Error completing request
*** Arguments: ('task(9tzplnj14g5w18d)', '1girl', '(bad-artist:1.0), (loli:1.2), (worst quality, low quality:1.4), (bad_prompt_version2:0.8), bad-hands-5 (bad-artist:1.0), (loli:1.2), (worst quality, low quality:1.4), EasyNegative, bad-hands-5,(monochrome:1.1), (greyscale),((((blurry background)))),orange shading,((((oversaturated,oversaturation)))),sunset,evening canvas frame, cartoon, 3d, ((disfigured)), ((bad art)), ((deformed)),((extra limbs)),((close up)),((b&w)), weird colors, blurry, (((duplicate))), ((morbid)), ((mutilated)), [out of frame], extra fingers, mutated hands, ((poorly drawn hands)), ((poorly drawn face)), (((mutation))), (((deformed))), ((ugly)), blurry, ((bad anatomy)), (((bad proportions))), ((extra limbs)), cloned face, (((disfigured))), out of frame, ugly, extra limbs, (bad anatomy), gross proportions, (malformed limbs), ((missing arms)), ((missing legs)), (((extra arms))), (((extra legs))), mutated hands, (fused fingers), (too many fingers), (((long neck))), Photoshop, video game, ugly, tiling, poorly drawn hands, poorly drawn feet, poorly drawn face, out of frame, mutation, mutated, extra limbs, extra legs, extra arms, disfigured, deformed, cross-eye, body out of frame, blurry, bad art, bad anatomy, 3d render', [], 20, 'Euler a', 1, 1, 7, 512, 512, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x00000205293A74C0>, 0, False, '', 0.8, -1, False, -1, 0, 0, 0, False, False, 'positive', 'comma', 0, False, False, 'start', '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False) {}
    Traceback (most recent call last):
      File "D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\modules\txt2img.py", line 55, in txt2img
        processed = processing.process_images(p)
      File "D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\modules\processing.py", line 736, in process_images
        res = process_images_inner(p)
      File "D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\modules\processing.py", line 841, in process_images_inner
        result = shared.sd_model(**kwargs)
    TypeError: 'OnnxRawPipeline' object is not callable

---

Additional information

I am using the 24.1.1 adrenaline drivers.

lshqqytiger commented 4 months ago

Please follow Extra instruction for DirectML users of https://github.com/lshqqytiger/stable-diffusion-webui-directml/discussions/149#discussioncomment-8392257 If you get same aten::scaled_dot_product_attention error, run these commands.

.\venv\Scripts\activate
pip install torch==2.2.0 torchvision --upgrade

waylaa commented 4 months ago

It works now, thank you. Are there any plans:

Automating those extra instructions during install

Custom VAE support because it gets reverted to 'automatic' and I get this in the console:

changing setting sd_vae to kl-f8-anime2.ckpt: AttributeError
Traceback (most recent call last):
File "D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\modules\options.py", line 146, in set
option.onchange()
File "D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\modules\call_queue.py", line 13, in f
res = func(*args, **kwargs)
File "D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\modules\initialize_util.py", line 175, in <lambda>
shared.opts.onchange("sd_vae", wrap_queued_call(lambda: sd_vae.reload_vae_weights()), call=False)
File "D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\modules\sd_vae.py", line 266, in reload_vae_weights
if sd_model.lowvram:
File "D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\diffusers\configuration_utils.py", line 142, in __getattr__
raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
AttributeError: 'OnnxRawPipeline' object has no attribute 'lowvram'

being able to use an infinite token length when using ONNX? Because currently I get this message in the console: The following part of your input was truncated because CLIP can only handle sequences up to 77 tokens:

lshqqytiger commented 4 months ago

we can't reinstall torch because Windows' permission system will block it.
fixed. I'm not sure Automatic will work, but .safetensors VAEs are supported.
doable. I'll take a look later.

waylaa commented 4 months ago

I don't know why but yesterday when i tried selecting a custom VAE, it reverted back to the 'automatic' option but now it doesn't. Instead, i get this after generating an image:

I don't if it's something serious but the images are generated just fine.

Traceback (most recent call last):
  File "D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\gradio\blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\gradio\blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\gradio\utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\modules\ui.py", line 1816, in <lambda>
    visible=shared.sd_model and shared.sd_model.cond_stage_key == "edit"
  File "D:\AI\Applications\Stable_Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\diffusers\configuration_utils.py", line 142, in __getattr__
    raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
AttributeError: 'OnnxStableDiffusionPipeline' object has no attribute 'cond_stage_key'

demon8555 commented 4 months ago

I'm not sure what did I wrong, but I get an error during the conversion. I downloaded an sdxl model (juggernautXL_v8Rundiffusion.safetensors), put it in under models\Stable-diffusion. on webui it is in the list at Stable Diffusion checkpoint. I select it, set 1024x1024. I set ONNX Stable Diffusion XL in settings for Diffusers pipeline. Here are the rest of the settings:

If I understand correctly, we don't have to manually convert the models after the recent updates. It does it when we first time generate something. But I get the following error:

*** Error completing request
*** Arguments: ('task(pujk0lj893wa9hi)', '', '', [], 20, 'PNDM', 1, 1, 7, 1024, 1024, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x000002489EE57B80>, 0, False, '', 0.8, -1, False, -1, 0, 0, 0, False, False, 'positive', 'comma', 0, False, False, 'start', '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False) {}
    Traceback (most recent call last):
      File "D:\apps\stable-diffusion-webui-directml\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "D:\apps\stable-diffusion-webui-directml\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "D:\apps\stable-diffusion-webui-directml\modules\txt2img.py", line 55, in txt2img
        processed = processing.process_images(p)
      File "D:\apps\stable-diffusion-webui-directml\modules\processing.py", line 736, in process_images
        res = process_images_inner(p)
      File "D:\apps\stable-diffusion-webui-directml\modules\processing.py", line 799, in process_images_inner
        shared.sd_model = preprocess_pipeline(p)
      File "D:\apps\stable-diffusion-webui-directml\modules\onnx_impl\__init__.py", line 190, in preprocess_pipeline
        shared.sd_model = shared.sd_model.preprocess(p)
      File "D:\apps\stable-diffusion-webui-directml\modules\onnx_impl\pipelines\__init__.py", line 342, in preprocess
        config.vae_sdxl_fp16_fix = self._is_sdxl and not shared.opts.diffusers_vae_upcast
      File "D:\apps\stable-diffusion-webui-directml\modules\options.py", line 123, in __getattr__
        return super(Options, self).__getattribute__(item)
    AttributeError: 'Options' object has no attribute 'diffusers_vae_upcast'

---

Edit: I tried both checkboxes toggling: (Olive use FP16 on optimization, Olive force FP32 for VAE Encoder)

waylaa commented 4 months ago

I'm not sure what did I wrong, but I get an error during the conversion. I downloaded an sdxl model (juggernautXL_v8Rundiffusion.safetensors), put it in under models\Stable-diffusion. on webui it is in the list at Stable Diffusion checkpoint. I select it, set 1024x1024. I set ONNX Stable Diffusion XL in settings for Diffusers pipeline. Here are the rest of the settings:

If I understand correctly, we don't have to manually convert the models after the recent updates. It does it when we first time generate something. But I get the following error:

*** Error completing request
*** Arguments: ('task(pujk0lj893wa9hi)', '', '', [], 20, 'PNDM', 1, 1, 7, 1024, 1024, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x000002489EE57B80>, 0, False, '', 0.8, -1, False, -1, 0, 0, 0, False, False, 'positive', 'comma', 0, False, False, 'start', '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False) {}
    Traceback (most recent call last):
      File "D:\apps\stable-diffusion-webui-directml\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "D:\apps\stable-diffusion-webui-directml\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "D:\apps\stable-diffusion-webui-directml\modules\txt2img.py", line 55, in txt2img
        processed = processing.process_images(p)
      File "D:\apps\stable-diffusion-webui-directml\modules\processing.py", line 736, in process_images
        res = process_images_inner(p)
      File "D:\apps\stable-diffusion-webui-directml\modules\processing.py", line 799, in process_images_inner
        shared.sd_model = preprocess_pipeline(p)
      File "D:\apps\stable-diffusion-webui-directml\modules\onnx_impl\__init__.py", line 190, in preprocess_pipeline
        shared.sd_model = shared.sd_model.preprocess(p)
      File "D:\apps\stable-diffusion-webui-directml\modules\onnx_impl\pipelines\__init__.py", line 342, in preprocess
        config.vae_sdxl_fp16_fix = self._is_sdxl and not shared.opts.diffusers_vae_upcast
      File "D:\apps\stable-diffusion-webui-directml\modules\options.py", line 123, in __getattr__
        return super(Options, self).__getattribute__(item)
    AttributeError: 'Options' object has no attribute 'diffusers_vae_upcast'

---

Edit: I tried both checkboxes toggling: (Olive use FP16 on optimization, Olive force FP32 for VAE Encoder)

try using FP32/FP16 on both. I had similar issues with FP16 optimizations and VAEs

demon8555 commented 4 months ago

Tried with both on, it's either the same error, or sometimes I get this:

ONNX: Failed to convert model: model='juggernautXL_v8Rundiffusion.safetensors', error=[WinError 3] The system cannot find the path specified: 'D:\\apps\\stable-diffusion-webui-directml\\models\\ONNX\\temp'

but the path exists

demon8555 commented 4 months ago

After today's commits, model conversion now works fine for me, thanks

waylaa commented 4 months ago

I get this new errors after today's commits and it occurs on any SD 1.5 model (I did not test with SDXL models because I do not have any):

I did not change any setting. Just opened the bat and tried to generate an image:

ONNX: Failed to load ONNX pipeline: is_sdxl=False
ONNX: You cannot load this model using the pipeline you selected. Please check Diffusers pipeline in ONNX Runtime Settings.
ONNX: processing=StableDiffusionProcessingTxt2Img, pipeline=OnnxRawPipeline

Traceback (most recent call last):
      File "D:\AI\Applications\Stable Diffusion (ONNX)\stable-diffusion-webui-directml\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "D:\AI\Applications\Stable Diffusion (ONNX)\stable-diffusion-webui-directml\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "D:\AI\Applications\Stable Diffusion (ONNX)\stable-diffusion-webui-directml\modules\txt2img.py", line 55, in txt2img
        processed = processing.process_images(p)
      File "D:\AI\Applications\Stable Diffusion (ONNX)\stable-diffusion-webui-directml\modules\processing.py", line 736, in process_images
        res = process_images_inner(p)
      File "D:\AI\Applications\Stable Diffusion (ONNX)\stable-diffusion-webui-directml\modules\processing.py", line 841, in process_images_inner
        result = shared.sd_model(**kwargs)
    TypeError: 'OnnxRawPipeline' object is not callable

demon8555 commented 4 months ago

I get this new errors after today's commits and it occurs on any SD 1.5 model (I did not test with SDXL models because I do not have any):

I did not change any setting. Just opened the bat and tried to generate an image:

ONNX: Failed to load ONNX pipeline: is_sdxl=False
ONNX: You cannot load this model using the pipeline you selected. Please check Diffusers pipeline in ONNX Runtime Settings.
ONNX: processing=StableDiffusionProcessingTxt2Img, pipeline=OnnxRawPipeline

Traceback (most recent call last):
      File "D:\AI\Applications\Stable Diffusion (ONNX)\stable-diffusion-webui-directml\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "D:\AI\Applications\Stable Diffusion (ONNX)\stable-diffusion-webui-directml\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "D:\AI\Applications\Stable Diffusion (ONNX)\stable-diffusion-webui-directml\modules\txt2img.py", line 55, in txt2img
        processed = processing.process_images(p)
      File "D:\AI\Applications\Stable Diffusion (ONNX)\stable-diffusion-webui-directml\modules\processing.py", line 736, in process_images
        res = process_images_inner(p)
      File "D:\AI\Applications\Stable Diffusion (ONNX)\stable-diffusion-webui-directml\modules\processing.py", line 841, in process_images_inner
        result = shared.sd_model(**kwargs)
    TypeError: 'OnnxRawPipeline' object is not callable

I would try to setup the project from scratch and see if it is working with minimal and default stuffs maybe...

Edit: okay actually SD 1.5 models working for me atm, but I get the same error for SDXL models...

Edit2: just figured out already, i have multiple project folders now (stable-diffusion-webui-directml) as I'm experimenting and troubleshooting. Looks like even if you are running from for example under stable-diffusion-webui-directml_new\, it might try to look for model files under an older stable-diffusion-webui-directml\ directory. Not sure why does it store absolute directory paths.

Edit3: yeah it stores absolute path: "onnx_cached_models_path": "D:\apps\stable-diffusion-webui-directml\models\ONNX\cache", "onnx_temp_dir": "D:\apps\stable-diffusion-webui-directml\models\ONNX\temp", So keep in mind if anyone out there copying around folders like me...

@waylaa Maybe check your console logs, as I had a file not found error before that same "'OnnxRawPipeline' object is not callable". error

waylaa commented 4 months ago

Re-installed and everything works. It would be nice if there was infinite prompt length support though. 77 token is really limiting.

Kylada commented 4 months ago

pip install torch==2.2.0 torchvision --upgrade

lshqqytiger, i am very very sorry, but after doing all you have said in Extra instruction for DirectML users and after pip install torch==2.2.0 torchvision --upgrade it now shows me this

(venv) (sd) I:\Stable Diffusion\stable-diffusion-webui-directml>webui-user.bat
venv "I:\Stable Diffusion\stable-diffusion-webui-directml\venv\Scripts\Python.exe"
fatal: No names found, cannot describe anything.
Python 3.10.6 | packaged by conda-forge | (main, Oct 24 2022, 16:02:16) [MSC v.1916 64 bit (AMD64)]
Version: 1.7.0
Commit hash: 7ed2ff1c35955af6a0ff37d0f8299fea82dc11d9
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
I:\Stable Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\pytorch_lightning\utilities\distributed.py:258: LightningDeprecationWarning: `pytorch_lightning.utilities.distributed.rank_zero_only` has been deprecated in v1.8.1 and will be removed in v2.0.0. You can import it from `pytorch_lightning.utilities` instead.
  rank_zero_deprecation(
Launching Web UI with arguments: --use-directml --use-cpu-torch
DirectML initialization failed: DLL load failed while importing torch_directml_native: Не найдена указанная процедура.
I:\Stable Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
Traceback (most recent call last):
  File "I:\Stable Diffusion\stable-diffusion-webui-directml\launch.py", line 48, in <module>
    main()
  File "I:\Stable Diffusion\stable-diffusion-webui-directml\launch.py", line 44, in main
    start()
  File "I:\Stable Diffusion\stable-diffusion-webui-directml\modules\launch_utils.py", line 677, in start
    import webui
  File "I:\Stable Diffusion\stable-diffusion-webui-directml\webui.py", line 13, in <module>
    initialize.imports()
  File "I:\Stable Diffusion\stable-diffusion-webui-directml\modules\initialize.py", line 34, in imports
    shared_init.initialize()
  File "I:\Stable Diffusion\stable-diffusion-webui-directml\modules\shared_init.py", line 28, in initialize
    directml_do_hijack()
  File "I:\Stable Diffusion\stable-diffusion-webui-directml\modules\dml\__init__.py", line 76, in directml_do_hijack
    if not torch.dml.has_float64_support(device):
  File "I:\Stable Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\torch\__init__.py", line 1932, in __getattr__
    raise AttributeError(f"module '{__name__}' has no attribute '{name}'")
AttributeError: module 'torch' has no attribute 'dml'

lshqqytiger commented 4 months ago

Do not use --use-directml with onnx.

Kylada commented 4 months ago

Now it gives me this:

(venv) (sd) I:\Stable Diffusion\stable-diffusion-webui-directml>webui-user.bat
venv "I:\Stable Diffusion\stable-diffusion-webui-directml\venv\Scripts\Python.exe"
fatal: No names found, cannot describe anything.
Python 3.10.6 | packaged by conda-forge | (main, Oct 24 2022, 16:02:16) [MSC v.1916 64 bit (AMD64)]
Version: 1.7.0
Commit hash: 7ed2ff1c35955af6a0ff37d0f8299fea82dc11d9
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
I:\Stable Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\pytorch_lightning\utilities\distributed.py:258: LightningDeprecationWarning: `pytorch_lightning.utilities.distributed.rank_zero_only` has been deprecated in v1.8.1 and will be removed in v2.0.0. You can import it from `pytorch_lightning.utilities` instead.
  rank_zero_deprecation(
Launching Web UI with arguments: --use-cpu-torch
Style database not found: I:\Stable Diffusion\stable-diffusion-webui-directml\styles.csv
Warning: caught exception 'Torch not compiled with CUDA enabled', memory monitor disabled
I:\Stable Diffusion\stable-diffusion-webui-directml\venv\lib\site-packages\diffusers\utils\outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
ONNX: selected=DmlExecutionProvider, available=['DmlExecutionProvider', 'CPUExecutionProvider']
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 6.3s (prepare environment: 13.1s, initialize shared: 2.0s, other imports: 0.1s, load scripts: 1.5s, create ui: 1.3s, gradio launch: 0.7s).
You have disabled the safety checker for <class 'diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion.StableDiffusionPipeline'> by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
Applying attention optimization: InvokeAI... done.
Olive implementation is experimental. It contains potentially an issue and is subject to change at any time.
*** Error completing request
*** Arguments: ('task(sjwzvj86j61nn88)', 'a dragon', '', [], 20, 'PNDM', 1, 1, 7, 512, 512, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x00000250807F09D0>, 0, False, '', 0.8, -1, False, -1, 0, 0, 0, False, False, 'positive', 'comma', 0, False, False, 'start', '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False) {}
    Traceback (most recent call last):
      File "I:\Stable Diffusion\stable-diffusion-webui-directml\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "I:\Stable Diffusion\stable-diffusion-webui-directml\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "I:\Stable Diffusion\stable-diffusion-webui-directml\modules\txt2img.py", line 55, in txt2img
        processed = processing.process_images(p)
      File "I:\Stable Diffusion\stable-diffusion-webui-directml\modules\processing.py", line 736, in process_images
        res = process_images_inner(p)
      File "I:\Stable Diffusion\stable-diffusion-webui-directml\modules\processing.py", line 799, in process_images_inner
        shared.sd_model = preprocess_pipeline(p)
      File "I:\Stable Diffusion\stable-diffusion-webui-directml\modules\onnx_impl\__init__.py", line 190, in preprocess_pipeline
        shared.sd_model = shared.sd_model.preprocess(p)
      File "I:\Stable Diffusion\stable-diffusion-webui-directml\modules\onnx_impl\pipelines\__init__.py", line 429, in preprocess
        pipeline = self.derive_properties(load_pipeline(self.constructor, out_dir, **kwargs))
      File "I:\Stable Diffusion\stable-diffusion-webui-directml\modules\onnx_impl\utils.py", line 127, in load_pipeline
        return cls(**patch_kwargs(cls, load_submodels(path, check_pipeline_sdxl(cls), load_init_dict(cls, path), **kwargs_ort)))
    TypeError: OnnxStableDiffusionPipeline.__init__() missing 4 required positional arguments: 'vae_encoder', 'vae_decoder', 'text_encoder', and 'unet'

lshqqytiger commented 4 months ago

If you have cached models in ./models/ONNX/cache, it may be corrupted. Remove and try again

Kylada commented 4 months ago

thanks. It now stops working without any errors shown during ort optimisation fusion, but it seems the problem here is in my extremely suboptimal hardware

derplayer commented 4 months ago

Please follow Extra instruction for DirectML users of #149 (comment) If you get same aten::scaled_dot_product_attention error, run these commands.
.\venv\Scripts\activate
pip install torch==2.2.0 torchvision --upgrade

I still had the 'aten::scaled_dot_product_attention' error, even after doing the extra DML step and this command. When you still have the error, for me, it helped to remove the venv folder, start the webui_user.bat once, close and redo all the commands. Seems something goes wrong with the pip cache sometimes?

Aelzaire commented 4 weeks ago

I'm not sure what did I wrong, but I get an error during the conversion. I downloaded an sdxl model (juggernautXL_v8Rundiffusion.safetensors), put it in under models\Stable-diffusion. on webui it is in the list at Stable Diffusion checkpoint. I select it, set 1024x1024. I set ONNX Stable Diffusion XL in settings for Diffusers pipeline. Here are the rest of the settings:

If I understand correctly, we don't have to manually convert the models after the recent updates. It does it when we first time generate something. But I get the following error:

*** Error completing request
*** Arguments: ('task(pujk0lj893wa9hi)', '', '', [], 20, 'PNDM', 1, 1, 7, 1024, 1024, False, 0.7, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x000002489EE57B80>, 0, False, '', 0.8, -1, False, -1, 0, 0, 0, False, False, 'positive', 'comma', 0, False, False, 'start', '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False) {}
    Traceback (most recent call last):
      File "D:\apps\stable-diffusion-webui-directml\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "D:\apps\stable-diffusion-webui-directml\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "D:\apps\stable-diffusion-webui-directml\modules\txt2img.py", line 55, in txt2img
        processed = processing.process_images(p)
      File "D:\apps\stable-diffusion-webui-directml\modules\processing.py", line 736, in process_images
        res = process_images_inner(p)
      File "D:\apps\stable-diffusion-webui-directml\modules\processing.py", line 799, in process_images_inner
        shared.sd_model = preprocess_pipeline(p)
      File "D:\apps\stable-diffusion-webui-directml\modules\onnx_impl\__init__.py", line 190, in preprocess_pipeline
        shared.sd_model = shared.sd_model.preprocess(p)
      File "D:\apps\stable-diffusion-webui-directml\modules\onnx_impl\pipelines\__init__.py", line 342, in preprocess
        config.vae_sdxl_fp16_fix = self._is_sdxl and not shared.opts.diffusers_vae_upcast
      File "D:\apps\stable-diffusion-webui-directml\modules\options.py", line 123, in __getattr__
        return super(Options, self).__getattribute__(item)
    AttributeError: 'Options' object has no attribute 'diffusers_vae_upcast'

---

Edit: I tried both checkboxes toggling: (Olive use FP16 on optimization, Olive force FP32 for VAE Encoder)

Having this same issue now on the latest commit, fresh install. Can generate in SD1.5 but not SDXL as it throws this 'diffusers_vae_upcast' error.

lshqqytiger / stable-diffusion-webui-amdgpu