search

NVIDIA / Stable-Diffusion-WebUI-TensorRT

TensorRT Extension for Stable Diffusion Web UI
MIT License
1.91k stars 146 forks source link

CUDA out of memory error (on 24GB 4090) #48

Closed Asinin3 closed 10 months ago

Asinin3 commented 1 year ago

I am using the hot_fix branch as I couldn't get the main branch to export. Receiving this error whenever I try and generate anything. tried using a static 512x512 engine, with batch size and count at 1. And it made no difference, it always tries to allocate 40.5 GB.

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 40.50 GiB (GPU 0; 23.99 GiB total capacity; 2.56 GiB already allocated; 19.69 GiB free; 2.62 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

contentis commented 1 year ago

Well, this is a new one. At what point does that error occur? Can you try restarting the webui?

Asinin3 commented 1 year ago

It seems to be related to the model. I changed to another model and exported the engine again and it wasn't trying to allocate as much VRAM. It's strange that the amount its trying to allocate doesn't seem to change when i change settings though.

contentis commented 1 year ago

During engine generation, the VRAM consumption might spike as it is trying different tactics, but is nothing to be concerned about. Once the engine is built and you generate images the required VRAM should be the size of weights + intermediate tensors depending on resolution and batch size.

Asinin3 commented 1 year ago

Re-opening as it happened again. OutOfMemoryError: CUDA out of memory. Tried to allocate 78.12 GiB (GPU 0; 23.99 GiB total capacity; 3.06 GiB already allocated Did one gen and it worked fine, left it for 15mins and did another and got that. Task manager showing only 4.1gb being used, so its not a memleak. Not sure why the amount its trying to allocate randomly increased so much.

MorkTheOrk commented 1 year ago

@Asinin3 do you have a stack trace?

Asinin3 commented 1 year ago

Log below. Note that I always use pretty similar settings, can see below that first error occured with batch size of 4, but i set it to 1 and ran it again and same thing except now showing vram is still allocated from the last run


Version: v1.6.0
Commit hash: 5ef669de080814067961f28357256e8fe27544f4
Requirement already satisfied: protobuf==3.20.2 in c:\users\someone\downloads\ai\sd.webui\system\python\lib\site-packages (3.20.2)
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: onnx-graphsurgeon in c:\users\someone\downloads\ai\sd.webui\system\python\lib\site-packages (0.3.27)
Requirement already satisfied: numpy in c:\users\someone\downloads\ai\sd.webui\system\python\lib\site-packages (from onnx-graphsurgeon) (1.23.5)
Requirement already satisfied: onnx in c:\users\someone\downloads\ai\sd.webui\system\python\lib\site-packages (from onnx-graphsurgeon) (1.14.1)
Requirement already satisfied: protobuf>=3.20.2 in c:\users\someone\downloads\ai\sd.webui\system\python\lib\site-packages (from onnx->onnx-graphsurgeon) (3.20.2)
Requirement already satisfied: typing-extensions>=3.6.2.1 in c:\users\someone\downloads\ai\sd.webui\system\python\lib\site-packages (from onnx->onnx-graphsurgeon) (4.8.0)
GS is not installed! Installing...
Installing protobuf
Installing onnx-graphsurgeon
You are up to date with the most recent release.
Launching Web UI with arguments: --autolaunch --update-check --xformers
[-] ADetailer initialized. version: 23.10.1, num models: 9
Loading weights [51020e5323] from C:\Users\Someone\Downloads\AI\sd.webui\webui\models\Stable-diffusion\mokoumixv1_prunedfp16.safetensors
Creating model from config: C:\Users\Someone\Downloads\AI\sd.webui\webui\configs\v1-inference.yaml
Loading VAE weights specified in settings: C:\Users\Someone\Downloads\AI\sd.webui\webui\models\VAE\klF8Anime2_klF8Anime2VAE.pt
Applying attention optimization: xformers... done.
Model loaded in 5.1s (load weights from disk: 0.7s, create model: 0.3s, apply weights to model: 2.9s, load VAE: 0.6s, calculate empty prompt: 0.4s).
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 23.9s (prepare environment: 6.7s, import torch: 2.6s, import gradio: 1.3s, setup paths: 1.9s, initialize shared: 0.3s, other imports: 1.3s, setup codeformer: 0.1s, load scripts: 4.2s, create ui: 5.2s, gradio launch: 0.4s).
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.6.0
Commit hash: 5ef669de080814067961f28357256e8fe27544f4
You are up to date with the most recent release.
Launching Web UI with arguments: --autolaunch --update-check --xformers
[-] ADetailer initialized. version: 23.10.1, num models: 9
Loading weights [51020e5323] from C:\Users\Someone\Downloads\AI\sd.webui\webui\models\Stable-diffusion\mokoumixv1_prunedfp16.safetensors
Creating model from config: C:\Users\Someone\Downloads\AI\sd.webui\webui\configs\v1-inference.yaml
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 9.4s (prepare environment: 1.7s, import torch: 1.8s, import gradio: 0.5s, setup paths: 0.4s, initialize shared: 0.1s, other imports: 0.3s, load scripts: 3.6s, create ui: 0.8s, gradio launch: 0.1s).
Loading VAE weights specified in settings: C:\Users\Someone\Downloads\AI\sd.webui\webui\models\VAE\klF8Anime2_klF8Anime2VAE.pt
Applying attention optimization: xformers... done.
Model loaded in 3.3s (load weights from disk: 0.5s, create model: 0.5s, apply weights to model: 1.5s, load VAE: 0.4s, calculate empty prompt: 0.3s).
*** Error completing request
*** Arguments: ('https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT', ['ads', 'localization', 'installed'], 0) {}    Traceback (most recent call last):
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\ui_extensions.py", line 411, in refresh_available_extensions
        available_extensions = json.loads(text)
      File "json\__init__.py", line 346, in loads
      File "json\decoder.py", line 337, in decode
      File "json\decoder.py", line 355, in raw_decode
    json.decoder.JSONDecodeError: Expecting value: line 8 column 1 (char 7)

---
*** Error completing request
*** Arguments: ('', 'https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT', '') {}
    Traceback (most recent call last):
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\ui_extensions.py", line 638, in <lambda>
        fn=modules.ui.wrap_gradio_call(lambda *args: [gr.update(), *install_extension_from_url(*args)], extra_outputs=[gr.update(), gr.update()]),
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\ui_extensions.py", line 359, in install_extension_from_url
        raise Exception(f'Extension with this URL is already installed: {url}')
    Exception: Extension with this URL is already installed: https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT

---
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.6.0
Commit hash: 5ef669de080814067961f28357256e8fe27544f4
You are up to date with the most recent release.
Launching Web UI with arguments: --autolaunch --update-check --xformers
[-] ADetailer initialized. version: 23.10.1, num models: 9
Loading weights [51020e5323] from C:\Users\Someone\Downloads\AI\sd.webui\webui\models\Stable-diffusion\mokoumixv1_prunedfp16.safetensors
Creating model from config: C:\Users\Someone\Downloads\AI\sd.webui\webui\configs\v1-inference.yaml
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 8.0s (prepare environment: 1.6s, import torch: 1.6s, import gradio: 0.5s, setup paths: 0.4s, initialize shared: 0.1s, other imports: 0.3s, load scripts: 2.9s, create ui: 0.3s, gradio launch: 0.2s).
Loading VAE weights specified in settings: C:\Users\Someone\Downloads\AI\sd.webui\webui\models\VAE\klF8Anime2_klF8Anime2VAE.pt
Applying attention optimization: xformers... done.
Model loaded in 3.0s (load weights from disk: 0.4s, create model: 0.4s, apply weights to model: 1.3s, load VAE: 0.6s, calculate empty prompt: 0.1s).
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.6.0
Commit hash: 5ef669de080814067961f28357256e8fe27544f4
You are up to date with the most recent release.
Launching Web UI with arguments: --autolaunch --update-check --xformers
[-] ADetailer initialized. version: 23.10.1, num models: 9
Loading weights [51020e5323] from C:\Users\Someone\Downloads\AI\sd.webui\webui\models\Stable-diffusion\mokoumixv1_prunedfp16.safetensors
Creating model from config: C:\Users\Someone\Downloads\AI\sd.webui\webui\configs\v1-inference.yaml
Loading VAE weights specified in settings: C:\Users\Someone\Downloads\AI\sd.webui\webui\models\VAE\klF8Anime2_klF8Anime2VAE.pt
Applying attention optimization: xformers... done.
Model loaded in 2.1s (load weights from disk: 0.3s, create model: 0.2s, apply weights to model: 1.1s, load VAE: 0.3s, calculate empty prompt: 0.1s).
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 9.9s (prepare environment: 1.8s, import torch: 1.8s, import gradio: 0.5s, setup paths: 0.4s, initialize shared: 0.2s, other imports: 0.3s, load scripts: 2.4s, create ui: 2.2s, gradio launch: 0.3s).
Reusing loaded model mokoumixv1_prunedfp16.safetensors [51020e5323] to load based64_v3.safetensors
Calculating sha256 for C:\Users\Someone\Downloads\AI\sd.webui\webui\models\Stable-diffusion\based64_v3.safetensors: 98a1428d4cc058a008f4723c7cea082678d3809dd322ee33e9b19b58c1cdf62f
Loading weights [98a1428d4c] from C:\Users\Someone\Downloads\AI\sd.webui\webui\models\Stable-diffusion\based64_v3.safetensors
Loading VAE weights specified in settings: C:\Users\Someone\Downloads\AI\sd.webui\webui\models\VAE\klF8Anime2_klF8Anime2VAE.pt
Applying attention optimization: xformers... done.
Weights loaded in 5.5s (send model to cpu: 1.8s, calculate hash: 1.7s, load weights from disk: 0.2s, apply weights to model: 0.5s, load VAE: 0.6s, move model to device: 0.7s).
Exporting based64_v3 to TensorRT
{'sample': [(1, 4, 96, 96), (2, 4, 128, 128), (8, 4, 128, 128)], 'timesteps': [(1,), (2,), (8,)], 'encoder_hidden_states': [(1, 77, 768), (2, 77, 768), (8, 154, 768)]}
No ONNX file found. Exporting ONNX...
Disabling attention optimization
ERROR:root:
Traceback (most recent call last):
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 74, in export_onnx
    inputs = modelobj.get_sample_input(
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\models.py", line 977, in get_sample_input
    latent_height, latent_width = self.check_dims(
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\models.py", line 263, in check_dims
    assert (
AssertionError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\anyio\to_thread.py", line 33, in run_sync    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 135, in export_unet_to_trt
    export_onnx(
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 129, in export_onnx
    exit()
  File "_sitebuiltins.py", line 26, in __call__
SystemExit: None
Exporting based64_v3 to TensorRT
{'sample': [(1, 4, 96, 96), (2, 4, 128, 128), (8, 4, 128, 128)], 'timesteps': [(1,), (2,), (8,)], 'encoder_hidden_states': [(1, 77, 768), (2, 77, 768), (8, 154, 768)]}
No ONNX file found. Exporting ONNX...
Disabling attention optimization
ERROR:root:
Traceback (most recent call last):
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 74, in export_onnx
    inputs = modelobj.get_sample_input(
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\models.py", line 977, in get_sample_input
    latent_height, latent_width = self.check_dims(
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\models.py", line 263, in check_dims
    assert (
AssertionError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 135, in export_unet_to_trt
    export_onnx(
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 129, in export_onnx
    exit()
  File "_sitebuiltins.py", line 26, in __call__
SystemExit: None
Exporting based64_v3 to TensorRT
{'sample': [(1, 4, 96, 96), (2, 4, 128, 128), (8, 4, 128, 128)], 'timesteps': [(1,), (2,), (8,)], 'encoder_hidden_states': [(1, 77, 768), (2, 77, 768), (8, 154, 768)]}
No ONNX file found. Exporting ONNX...
Disabling attention optimization
ERROR:root:
Traceback (most recent call last):
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 74, in export_onnx
    inputs = modelobj.get_sample_input(
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\models.py", line 977, in get_sample_input
    latent_height, latent_width = self.check_dims(
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\models.py", line 263, in check_dims
    assert (
AssertionError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 135, in export_unet_to_trt
    export_onnx(
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 129, in export_onnx
    exit()
  File "_sitebuiltins.py", line 26, in __call__
SystemExit: None
Exporting based64_v3 to TensorRT
{'sample': [(1, 4, 96, 96), (2, 4, 128, 128), (8, 4, 128, 128)], 'timesteps': [(1,), (2,), (8,)], 'encoder_hidden_states': [(1, 77, 768), (2, 77, 768), (8, 154, 768)]}
No ONNX file found. Exporting ONNX...
Disabling attention optimization
ERROR:root:
Traceback (most recent call last):
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 74, in export_onnx
    inputs = modelobj.get_sample_input(
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\models.py", line 977, in get_sample_input
    latent_height, latent_width = self.check_dims(
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\models.py", line 263, in check_dims
    assert (
AssertionError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 135, in export_unet_to_trt
    export_onnx(
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 129, in export_onnx
    exit()
  File "_sitebuiltins.py", line 26, in __call__
SystemExit: None
Exporting based64_v3 to TensorRT
{'sample': [(1, 4, 96, 96), (2, 4, 128, 128), (8, 4, 128, 128)], 'timesteps': [(1,), (2,), (8,)], 'encoder_hidden_states': [(1, 77, 768), (2, 77, 768), (8, 154, 768)]}
No ONNX file found. Exporting ONNX...
Disabling attention optimization
ERROR:root:
Traceback (most recent call last):
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 74, in export_onnx
    inputs = modelobj.get_sample_input(
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\models.py", line 977, in get_sample_input
    latent_height, latent_width = self.check_dims(
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\models.py", line 263, in check_dims
    assert (
AssertionError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 135, in export_unet_to_trt
    export_onnx(
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 129, in export_onnx
    exit()
  File "_sitebuiltins.py", line 26, in __call__
SystemExit: None
Exporting based64_v3 to TensorRT
{'sample': [(1, 4, 96, 96), (2, 4, 128, 128), (8, 4, 128, 128)], 'timesteps': [(1,), (2,), (8,)], 'encoder_hidden_states': [(1, 77, 768), (2, 77, 768), (8, 154, 768)]}
No ONNX file found. Exporting ONNX...
Disabling attention optimization
ERROR:root:
Traceback (most recent call last):
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 74, in export_onnx
    inputs = modelobj.get_sample_input(
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\models.py", line 977, in get_sample_input
    latent_height, latent_width = self.check_dims(
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\models.py", line 263, in check_dims
    assert (
AssertionError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\gradio\utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 135, in export_unet_to_trt
    export_onnx(
  File "C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 129, in export_onnx
    exit()
  File "_sitebuiltins.py", line 26, in __call__
SystemExit: None
Exporting based64_v3 to TensorRT
{'sample': [(1, 4, 64, 64), (2, 4, 64, 64), (8, 4, 96, 96)], 'timesteps': [(1,), (2,), (8,)], 'encoder_hidden_states': [(1, 77, 768), (2, 77, 768), (8, 154, 768)]}
No ONNX file found. Exporting ONNX...
Disabling attention optimization
C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\einops\einops.py:314: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  known = {axis for axis in composite_axis if axis_name2known_length[axis] != _unknown_axis_length}
C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\einops\einops.py:315: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  unknown = {axis for axis in composite_axis if axis_name2known_length[axis] == _unknown_axis_length}
C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py:158: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert x.shape[1] == self.channels
C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\sd_hijack_unet.py:26: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if a.shape[-2:] != b.shape[-2:]:
C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py:109: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  assert x.shape[1] == self.channels
============= Diagnostic Run torch.onnx.export version 2.0.1+cu118 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

[W] 'colored' module is not installed, will not use colors when logging. To enable colors, please install the 'colored' module: python3 -m pip install colored
[E] ONNX-Runtime is not installed, so constant folding may be suboptimal or not work at all.
    Consider installing ONNX-Runtime: C:\Users\Someone\Downloads\AI\sd.webui\system\python\python.exe -m pip install onnxruntime
[I] Folding Constants | Pass 1
[!] Module: 'onnxruntime.tools.symbolic_shape_infer' is required but could not be imported.
    Note: Error was: No module named 'onnxruntime'
    You can set POLYGRAPHY_AUTOINSTALL_DEPS=1 in your environment variables to allow Polygraphy to automatically install missing modules.
[W] Falling back to `onnx.shape_inference` because `onnxruntime.tools.symbolic_shape_infer` either could not be loaded or did not run successfully.
    Note that using ONNX-Runtime for shape inference may be faster and require less memory.
    Consider installing ONNX-Runtime or setting POLYGRAPHY_AUTOINSTALL_DEPS=1 in your environment variables to allow Polygraphy to do so automatically.
[W] Attempting to run shape inference on a large model (1641.0 MiB). This may require a large amount of memory.
    If memory consumption becomes too high, the process may be killed. You may want to try disabling shape inference in that case.
[I]     Total Nodes | Original:  8992, After Folding:  6216 |  2776 Nodes Folded
[I] Folding Constants | Pass 2
[W] colored module is not installed, will not use colors when logging. To enable colors, please install the colored module: python3 -m pip install colored
[W] Inference failed. You may want to try enabling partitioning to see better results. Note: Error was:
No module named 'onnxruntime'
[!] Module: 'onnxruntime.tools.symbolic_shape_infer' is required but could not be imported.
    Note: Error was: No module named 'onnxruntime'
    You can set POLYGRAPHY_AUTOINSTALL_DEPS=1 in your environment variables to allow Polygraphy to automatically install missing modules.
[W] Attempting to run shape inference on a large model (1642.0 MiB). This may require a large amount of memory.
    If memory consumption becomes too high, the process may be killed. You may want to try disabling shape inference in that case.
[I]     Total Nodes | Original:  6216, After Folding:  6152 |    64 Nodes Folded
[I] Folding Constants | Pass 3
[W] colored module is not installed, will not use colors when logging. To enable colors, please install the colored module: python3 -m pip install colored
[W] Inference failed. You may want to try enabling partitioning to see better results. Note: Error was:
No module named 'onnxruntime'
[!] Module: 'onnxruntime.tools.symbolic_shape_infer' is required but could not be imported.
    Note: Error was: No module named 'onnxruntime'
    You can set POLYGRAPHY_AUTOINSTALL_DEPS=1 in your environment variables to allow Polygraphy to automatically install missing modules.
[I]     Total Nodes | Original:  6152, After Folding:  6152 |     0 Nodes Folded
Exported to ONNX.
Building TensorRT engine... This can take a while, please check the progress in the terminal.
Building TensorRT engine for C:\Users\Someone\Downloads\AI\sd.webui\webui\models\Unet-onnx\based64_v3_aa59e63c.onnx: C:\Users\Someone\Downloads\AI\sd.webui\webui\models\Unet-trt\based64_v3_aa59e63c_cc89_sample=1x4x64x64+2x4x64x64+8x4x96x96-timesteps=1+2+8-encoder_hidden_states=1x77x768+2x77x768+8x154x768.trt
[libprotobuf WARNING **************************************************************************\externals\protobuf\3.0.0\src\google\protobuf\io\coded_stream.cc:604] Reading dangerously large protocol message.  If the message turns out to be larger than 2147483647 bytes, parsing will be halted for security reasons.  To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
[libprotobuf WARNING **************************************************************************\externals\protobuf\3.0.0\src\google\protobuf\io\coded_stream.cc:81] The total number of bytes read was 1721931702
[libprotobuf WARNING **************************************************************************\externals\protobuf\3.0.0\src\google\protobuf\io\coded_stream.cc:604] Reading dangerously large protocol message.  If the message turns out to be larger than 2147483647 bytes, parsing will be halted for security reasons.  To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
[libprotobuf WARNING **************************************************************************\externals\protobuf\3.0.0\src\google\protobuf\io\coded_stream.cc:81] The total number of bytes read was 1721931702
[I] Loading tactic timing cache from C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\timing_caches\timing_cache_win_cc89.cache
[I] Building engine with configuration:
    Flags                  | [FP16, REFIT, TF32]
    Engine Capability      | EngineCapability.DEFAULT
    Memory Pools           | [WORKSPACE: 24563.50 MiB, TACTIC_DRAM: 24563.50 MiB]
    Tactic Sources         | [CUBLAS, CUDNN, EDGE_MASK_CONVOLUTIONS, JIT_CONVOLUTIONS]
    Profiling Verbosity    | ProfilingVerbosity.LAYER_NAMES_ONLY
    Preview Features       | [FASTER_DYNAMIC_SHAPES_0805, DISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805]
Building engine:  50%|█████████████████████████████████████████████████████████████████                                                                 | 3/6 [00:00<00:00, 11.94it/s][W] Cache result detected as invalid for node: /input_blocks.4/input_blocks.4.0/skip_connection/Conv, LayerImpl: CaskGemmConvolution, tactic: 0x0000000204040190 | 0/1 [00:00<?, ?it/s]
Building engine: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6/6 [01:26<00:00, 14.43s/it]
[I] Finished engine building in 87.893 seconds
[I] Saving tactic timing cache to C:\Users\Someone\Downloads\AI\sd.webui\webui\extensions\Stable-Diffusion-WebUI-TensorRT\timing_caches\timing_cache_win_cc89.cache
[I] Saving engine to C:\Users\Someone\Downloads\AI\sd.webui\webui\models\Unet-trt\based64_v3_aa59e63c_cc89_sample=1x4x64x64+2x4x64x64+8x4x96x96-timesteps=1+2+8-encoder_hidden_states=1x77x768+2x77x768+8x154x768.trt
TensorRT engines has been saved to disk.
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:06<00:00,  4.41it/s]
  0%|                                                                                                                                                          | 0/30 [00:20<?, ?it/s]
*** Error completing request
*** Arguments: ('task(ietba8bt1748h0f)', 'race', '(worst quality, low quality:1.4)', [], 30, 'DPM++ 2M Karras', 1, 4, 8, 512, 512, True, 0.6, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x000001DCCA263700>, 0, False, '', 0.8, 3313499687, False, -1, 0, 0, 0, False, False, {'ad_model': 'face_yolov8n.pt', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'inpaint_global_harmonious', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, {'ad_model': 'None', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'inpaint_global_harmonious', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False) {}
    Traceback (most recent call last):
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\txt2img.py", line 55, in txt2img
        processed = processing.process_images(p)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\processing.py", line 732, in process_images
        res = process_images_inner(p)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\processing.py", line 867, in process_images_inner
        samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\processing.py", line 1156, in sample
        return self.sample_hr_pass(samples, decoded_samples, seeds, subseeds, subseed_strength, prompts)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\processing.py", line 1242, in sample_hr_pass
        samples = self.sampler.sample_img2img(self, samples, noise, self.hr_c, self.hr_uc, steps=self.hr_second_pass_steps or self.steps, image_conditioning=image_conditioning)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\sd_samplers_kdiffusion.py", line 188, in sample_img2img
        samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\sd_samplers_common.py", line 261, in launch_sampling
        return func()
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\sd_samplers_kdiffusion.py", line 188, in <lambda>
        samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\k-diffusion\k_diffusion\sampling.py", line 594, in sample_dpmpp_2m
        denoised = model(x, sigmas[i] * s_in, **extra_args)
      File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\sd_samplers_cfg_denoiser.py", line 169, in forward
        x_out = self.inner_model(x_in, sigma_in, cond=make_condition_dict(cond_in, image_cond_in))
      File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\k-diffusion\k_diffusion\external.py", line 112, in forward
        eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\k-diffusion\k_diffusion\external.py", line 138, in get_eps
        return self.inner_model.apply_model(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\sd_hijack_utils.py", line 17, in <lambda>
        setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\sd_hijack_utils.py", line 28, in __call__
        return self.__orig_func(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 858, in apply_model
        x_recon = self.model(x_noisy, t, **cond)
      File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 1335, in forward
        out = self.diffusion_model(x, t, context=cc)
      File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\sd_unet.py", line 91, in UNetModel_forward
        return ldm.modules.diffusionmodules.openaimodel.copy_of_UNetModel_forward_for_webui(self, x, timesteps, context, *args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 797, in forward
        h = module(h, emb, context)
      File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 84, in forward
        x = layer(x, context)
      File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\stable-diffusion-stability-ai\ldm\modules\attention.py", line 334, in forward
        x = block(x, context=context[i])
      File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\stable-diffusion-stability-ai\ldm\modules\attention.py", line 269, in forward
        return checkpoint(self._forward, (x, context), self.parameters(), self.checkpoint)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\util.py", line 123, in checkpoint
        return func(*inputs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\stable-diffusion-stability-ai\ldm\modules\attention.py", line 272, in _forward
        x = self.attn1(self.norm1(x), context=context if self.disable_self_attn else None) + x
      File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\hypernetworks\hypernetwork.py", line 393, in attention_CrossAttention_forward
        sim = einsum('b i d, b j d -> b i j', q, k) * self.scale
    torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 32.00 GiB (GPU 0; 23.99 GiB total capacity; 34.67 GiB already allocated; 0 bytes free; 34.77 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

---
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:02<00:00, 11.65it/s]
  0%|                                                                                                                                                          | 0/30 [00:05<?, ?it/s]
*** Error completing request
*** Arguments: ('task(xcm5kjusg2tkviy)', 'race', '(worst quality, low quality:1.4)', [], 30, 'DPM++ 2M Karras', 1, 1, 8, 512, 512, True, 0.6, 2, 'Latent', 0, 0, 0, 'Use same checkpoint', 'Use same sampler', '', '', [], <gradio.routes.Request object at 0x000001DC0B4F6680>, 0, False, '', 0.8, 3313499687, False, -1, 0, 0, 0, False, False, {'ad_model': 'face_yolov8n.pt', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'inpaint_global_harmonious', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, {'ad_model': 'None', 'ad_prompt': '', 'ad_negative_prompt': '', 'ad_confidence': 0.3, 'ad_mask_k_largest': 0, 'ad_mask_min_ratio': 0, 'ad_mask_max_ratio': 1, 'ad_x_offset': 0, 'ad_y_offset': 0, 'ad_dilate_erode': 4, 'ad_mask_merge_invert': 'None', 'ad_mask_blur': 4, 'ad_denoising_strength': 0.4, 'ad_inpaint_only_masked': True, 'ad_inpaint_only_masked_padding': 32, 'ad_use_inpaint_width_height': False, 'ad_inpaint_width': 512, 'ad_inpaint_height': 512, 'ad_use_steps': False, 'ad_steps': 28, 'ad_use_cfg_scale': False, 'ad_cfg_scale': 7, 'ad_use_checkpoint': False, 'ad_checkpoint': 'Use same checkpoint', 'ad_use_vae': False, 'ad_vae': 'Use same VAE', 'ad_use_sampler': False, 'ad_sampler': 'DPM++ 2M Karras', 'ad_use_noise_multiplier': False, 'ad_noise_multiplier': 1, 'ad_use_clip_skip': False, 'ad_clip_skip': 1, 'ad_restore_face': False, 'ad_controlnet_model': 'None', 'ad_controlnet_module': 'inpaint_global_harmonious', 'ad_controlnet_weight': 1, 'ad_controlnet_guidance_start': 0, 'ad_controlnet_guidance_end': 1, 'is_api': ()}, False, False, 'positive', 'comma', 0, False, False, '', 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, False) {}
    Traceback (most recent call last):
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\call_queue.py", line 57, in f
        res = list(func(*args, **kwargs))
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\call_queue.py", line 36, in f
        res = func(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\txt2img.py", line 55, in txt2img
        processed = processing.process_images(p)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\processing.py", line 732, in process_images
        res = process_images_inner(p)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\processing.py", line 867, in process_images_inner
        samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\processing.py", line 1156, in sample
        return self.sample_hr_pass(samples, decoded_samples, seeds, subseeds, subseed_strength, prompts)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\processing.py", line 1242, in sample_hr_pass
        samples = self.sampler.sample_img2img(self, samples, noise, self.hr_c, self.hr_uc, steps=self.hr_second_pass_steps or self.steps, image_conditioning=image_conditioning)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\sd_samplers_kdiffusion.py", line 188, in sample_img2img
        samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\sd_samplers_common.py", line 261, in launch_sampling
        return func()
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\sd_samplers_kdiffusion.py", line 188, in <lambda>
        samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
      File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
        return func(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\k-diffusion\k_diffusion\sampling.py", line 594, in sample_dpmpp_2m
        denoised = model(x, sigmas[i] * s_in, **extra_args)
      File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\sd_samplers_cfg_denoiser.py", line 169, in forward
        x_out = self.inner_model(x_in, sigma_in, cond=make_condition_dict(cond_in, image_cond_in))
      File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\k-diffusion\k_diffusion\external.py", line 112, in forward
        eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\k-diffusion\k_diffusion\external.py", line 138, in get_eps
        return self.inner_model.apply_model(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\sd_hijack_utils.py", line 17, in <lambda>
        setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\sd_hijack_utils.py", line 28, in __call__
        return self.__orig_func(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 858, in apply_model
        x_recon = self.model(x_noisy, t, **cond)
      File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 1335, in forward
        out = self.diffusion_model(x, t, context=cc)
      File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\sd_unet.py", line 91, in UNetModel_forward
        return ldm.modules.diffusionmodules.openaimodel.copy_of_UNetModel_forward_for_webui(self, x, timesteps, context, *args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 797, in forward
        h = module(h, emb, context)
      File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\openaimodel.py", line 84, in forward
        x = layer(x, context)
      File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\stable-diffusion-stability-ai\ldm\modules\attention.py", line 334, in forward
        x = block(x, context=context[i])
      File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\stable-diffusion-stability-ai\ldm\modules\attention.py", line 269, in forward
        return checkpoint(self._forward, (x, context), self.parameters(), self.checkpoint)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\stable-diffusion-stability-ai\ldm\modules\diffusionmodules\util.py", line 123, in checkpoint
        return func(*inputs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\repositories\stable-diffusion-stability-ai\ldm\modules\attention.py", line 272, in _forward
        x = self.attn1(self.norm1(x), context=context if self.disable_self_attn else None) + x
      File "C:\Users\Someone\Downloads\AI\sd.webui\system\python\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
        return forward_call(*args, **kwargs)
      File "C:\Users\Someone\Downloads\AI\sd.webui\webui\modules\hypernetworks\hypernetwork.py", line 402, in attention_CrossAttention_forward
        attn = sim.softmax(dim=-1)
    torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 16.00 GiB (GPU 0; 23.99 GiB total capacity; 10.22 GiB already allocated; 0 bytes free; 26.22 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

---```
contentis commented 1 year ago

Okay, there seem to be a bunch of assertions failing before we even get to the export. Have you had the Auto1111 TensorRT extension installed before? I suggest going to the extension folder and deleting any TensorRT-related extension. Launch the webui and add the extension back. You can also install it from the Available Extension tab now: image

Asinin3 commented 1 year ago

I had Auto's Tensor RT extension installed for a few mins last week and couldn't get it working so I removed it, and deleted the files. I don't think that there's anything left over causing issues.

doomhanner commented 1 year ago

I'm having the same issue, with similar context. -3090. -I had tried installing the extension, and it failed, as A1111 was out of date. Deleted the folder for the extension, updated A1111, then reinstalled the extension. -I now get out of VRAM errors any time I try to build an engine (default or otherwise).

doomhanner commented 1 year ago

I'd like to add that I can now build default and custom engines after updating to the 10/31 drivers that include system memory overflow for CUDA. I have that feature enabled, but don't know if that's what made the difference.

Shrkbait2 commented 10 months ago

@doomhanner > I'd like to add that I can now build default and custom engines after updating to the 10/31 drivers that include system memory overflow for CUDA. I have that feature enabled, but don't know if that's what made the difference. Which driver are you referring to?

Milor123 commented 7 months ago

I have the same problem. I can built the tensor file, but when try generate a image in SDXL with it enabled, it shows OutOfMemoryError: CUDA out of memory, I have a 4070