Exporting to ONNX failed

[Eprixko]

Hey guys, when I click on Export Default Engine, I get this error after few seconds, anyone knows how to fix it?

polygraphy.exception.exception.PolygraphyException: Failed to parse ONNX model. Does the model file exist and contain a valid ONNX model?
Disabling attention optimization
Exporting juggernautXL_v8Rundiffusion to TensorRT using - Batch Size: 1-1-1
Height: 1024-1024-1024
Width: 1024-1024-1024
Token Count: 77-77-77
Disabling attention optimization
============= Diagnostic Run torch.onnx.export version 2.0.1+cu118 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

ERROR:root:Exporting to ONNX failed. Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)

[T9es]

List your webUI start arguments.

[Eprixko]

List your webUI start arguments.

set COMMANDLINE_ARGS=--xformers --no-half-vae --api

[T9es]

Have you tried specifying the device ID?

There's a lot of variables, what hardware are you using? Are you running on a dedicated GPU?

Also, --api might cause the model.json not updating. This is minor, leave the issue for now if this doesn't bother you, or it doesn't affect you.

[T9es]

Small check, can you check your memory usage when exporting the model to ONNX?

I just discovered that exporting to ONNX is eating up all 24gb of my vram, and then 14-16gb of my ram. That overflow to ram might be the root issue of this. I can't seem to export a model as well currently, using the static shapes.

[bigmover]

Hey guys, when I click on Export Default Engine, I get this error after few seconds, anyone knows how to fix it?

polygraphy.exception.exception.PolygraphyException: Failed to parse ONNX model. Does the model file exist and contain a valid ONNX model?
Disabling attention optimization
Exporting juggernautXL_v8Rundiffusion to TensorRT using - Batch Size: 1-1-1
Height: 1024-1024-1024
Width: 1024-1024-1024
Token Count: 77-77-77
Disabling attention optimization
============= Diagnostic Run torch.onnx.export version 2.0.1+cu118 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

ERROR:root:Exporting to ONNX failed. Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)

Hi guys! Are you enabling the extension on linux？

[Eprixko]

Have you tried specifying the device ID?

There's a lot of variables, what hardware are you using? Are you running on a dedicated GPU?

Also, --api might cause the model.json not updating. This is minor, leave the issue for now if this doesn't bother you, or it doesn't affect you.

Yes, I tried to specify the device id by --device-id 0, the error was still the same.

I'm using MSI GF75 Thin 10UE-065CZ: Specs are: Intel Core i7 10750H Comet Lake with 6cores 2.6GHz (Core Boost to 5GHz), 16GB RAM DDR4 and NVIDIA GeForce RTX 3060 6GB

Just tried removing the --api and it didn't help, still same error.

Small check, can you check your memory usage when exporting the model to ONNX?

I just discovered that exporting to ONNX is eating up all 24gb of my vram, and then 14-16gb of my ram. That overflow to ram might be the root issue of this. I can't seem to export a model as well currently, using the static shapes.

I don't know how to check that and if it's even possible as the error Exporting to ONNX failed pops up like 5 seconds after I click on Export Default Engine.

Is it possible that the error pops up because I'm running Automatic1111 trough Pinokio?

[Eprixko]

By the way, here is the whole thing what happens after I click on Export Default Engine.

Exporting juggernautXL_v8Rundiffusion to TensorRT using - Batch Size: 1-1-1
Height: 1024-1024-1024
Width: 1024-1024-1024
Token Count: 75-75-75
Disabling attention optimization
============= Diagnostic Run torch.onnx.export version 2.0.1+cu118 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

ERROR:root:Exporting to ONNX failed. Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)   
Building TensorRT engine... This can take a while, please check the progress in the terminal.
Building TensorRT engine for C:\pinokio\api\Automatic1111\app\models\Unet-onnx\juggernautXL_v8Rundiffusion.onnx: C:\pinokio\api\Automatic1111\app\models\Unet-trt\juggernautXL_v8Rundiffusion_e80db5ed_cc86_sample=2x4x128x128+2x4x128x128+2x4x128x128-timesteps=2+2+2-encoder_hidden_states=2x77x2048+2x77x2048+2x77x2048-y=2x2816+2x2816+2x2816.trt
Could not open file C:\pinokio\api\Automatic1111\app\models\Unet-onnx\juggernautXL_v8Rundiffusion.onnx
Could not open file C:\pinokio\api\Automatic1111\app\models\Unet-onnx\juggernautXL_v8Rundiffusion.onnx 
[W] 'colored' module is not installed, will not use colors when logging. To enable colors, please install the 'colored' module: python3 -m pip install colored
[E] ModelImporter.cpp:773: Failed to parse ONNX model from file: C:\pinokio\api\Automatic1111\app\models\Unet-onnx\juggernautXL_v8Rundiffusion.onnx
[!] Failed to parse ONNX model. Does the model file exist and contain a valid ONNX model?
Traceback (most recent call last):
  File "C:\pinokio\api\Automatic1111\app\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\pinokio\api\Automatic1111\app\venv\lib\site-packages\gradio\blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "C:\pinokio\api\Automatic1111\app\venv\lib\site-packages\gradio\blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\pinokio\api\Automatic1111\app\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\pinokio\api\Automatic1111\app\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "C:\pinokio\api\Automatic1111\app\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "C:\pinokio\api\Automatic1111\app\venv\lib\site-packages\gradio\utils.py", line 707, in wrapper 
    response = f(*args, **kwargs)
  File "C:\pinokio\api\Automatic1111\app\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 126, in export_unet_to_trt
    ret = export_trt(
  File "C:\pinokio\api\Automatic1111\app\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 231, in export_trt
    ret = engine.build(
  File "C:\pinokio\api\Automatic1111\app\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 227, in build
    network = network_from_onnx_path(
  File "<string>", line 3, in network_from_onnx_path
  File "C:\pinokio\api\Automatic1111\app\venv\lib\site-packages\polygraphy\backend\base\loader.py", line 40, in __call__
    return self.call_impl(*args, **kwargs)
  File "C:\pinokio\api\Automatic1111\app\venv\lib\site-packages\polygraphy\util\util.py", line 710, in wrapped
    return func(*args, **kwargs)
  File "C:\pinokio\api\Automatic1111\app\venv\lib\site-packages\polygraphy\backend\trt\loader.py", line 227, in call_impl
    trt_util.check_onnx_parser_errors(parser, success)
  File "C:\pinokio\api\Automatic1111\app\venv\lib\site-packages\polygraphy\backend\trt\util.py", line 86, in check_onnx_parser_errors
    G_LOGGER.critical("Failed to parse ONNX model. Does the model file exist and contain a valid ONNX model?")
  File "C:\pinokio\api\Automatic1111\app\venv\lib\site-packages\polygraphy\logger\logger.py", line 605, in critical
    raise ExceptionType(message) from None
polygraphy.exception.exception.PolygraphyException: Failed to parse ONNX model. Does the model file exist and contain a valid ONNX model?

[Theliel]

Since the last updates, it consumes a huge amount of memory, making the extension basically useless. In the previous version, it adapted itself until it could allocate an available amount. Right now, it tries to put all that it requires in VRAM, if there is not enough in RAM, and if it is not enough it ends with the error shown.

Previously, you could easily create profiles for 1024x1024 x2 or 768x768 x8. Now, this is totally impossible, not even 768x768 x3

[StanleyPain]

I'm getting the same error, or this one:

ERROR:root:Exporting to ONNX failed. 'NoneType' object has no attribute 'shape'

Only seems to happen with SDXL models for me.

[LunaMorbus]

I am using a laptop and the graphics card is RTX4060.

Yes @.***> 于 2024年1月29日周一 上午6:13写道：

Have you tried specifying the device ID?

There's a lot of variables, what hardware are you using? Are you running on a dedicated GPU?

Also, --api might cause the model.json not updating. This is minor, leave the issue for now if this doesn't bother you, or it doesn't affect you.

— Reply to this email directly, view it on GitHub https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT/issues/262#issuecomment-1913738714, or unsubscribe https://github.com/notifications/unsubscribe-auth/A35V3ORWVDMAFLEZY6BW5RTYQ3ERPAVCNFSM6AAAAABCNBIQCWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJTG4ZTQNZRGQ . You are receiving this because you are subscribed to this thread.Message ID: @.*** com>

[LunaMorbus]

No, I am using win11

Jack @.***> 于 2024年1月29日周一 上午10:03写道：

Hey guys, when I click on Export Default Engine, I get this error after few seconds, anyone knows how to fix it?

polygraphy.exception.exception.PolygraphyException: Failed to parse ONNX model. Does the model file exist and contain a valid ONNX model? Disabling attention optimization Exporting juggernautXL_v8Rundiffusion to TensorRT using - Batch Size: 1-1-1 Height: 1024-1024-1024 Width: 1024-1024-1024 Token Count: 77-77-77 Disabling attention optimization ============= Diagnostic Run torch.onnx.export version 2.0.1+cu118 ============= verbose: False, log level: Level.ERROR ======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

ERROR:root:Exporting to ONNX failed. Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)

Hi guys! Are you enabling the extension on linux？

— Reply to this email directly, view it on GitHub https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT/issues/262#issuecomment-1913839789, or unsubscribe https://github.com/notifications/unsubscribe-auth/A35V3OVGROA5P7Z7S6TZNCDYQ37QLAVCNFSM6AAAAABCNBIQCWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJTHAZTSNZYHE . You are receiving this because you are subscribed to this thread.Message ID: @.*** com>

[uni-devboy]

None of the above worked for me, this is what worked for me and fixed it to export engines with SD Forge https://github.com/lllyasviel/stable-diffusion-webui-forge/issues/64#issuecomment-2001987142

[wobba]

None of the above worked for me, this is what worked for me and fixed it to export engines with SD Forge lllyasviel/stable-diffusion-webui-forge#64 (comment)

This together with --always-gpu solved it for me