Tensorrt cannot appear on the webui

[GitGod76]

I get this error

Error loading script: trt.py

Traceback (most recent call last):

File "F:\stable-diffusion-webui\modules[scripts.py](https://scripts.py/)", line 263, in load_scripts

script_module = script_loading.load_module(scriptfile.path)

File "F:\stable-diffusion-webui\modules\script_loading.py", line 10, in load_module

module_spec.loader.exec_module(module)

File "", line 883, in exec_module

File "", line 241, in _call_with_frames_removed

File "F:\stable-diffusion-webui\extensions\stable-diffusion-webui-tensorrt\scripts[trt.py](https://trt.py/)", line 6, in

from modules import script_callbacks, sd_unet, devices, shared, paths_internal

ImportError: cannot import name 'sd_unet' from 'modules' (unknown location)

[JilekJosef]

If it writes something about pycuda too than first install https://visualstudio.microsoft.com/cs/visual-cpp-build-tools/ so pycuda can be installed. Also you need to switch to dev branch for webui by: git checkout dev

[GitGod76]

I will try it, thank you

[fredrickflower]

Same error here as well

[MoreColors123]

If it writes something about pycuda too than first install https://visualstudio.microsoft.com/cs/visual-cpp-build-tools/ so pycuda can be installed. Also you need to switch to dev branch for webui by: git checkout dev

thank you. when installing build tools, what needs to be enabled? there is a ton of options there.

[JilekJosef]

Just like this

[MoreColors123]

thank you, this was exactly what was needed. it installs fine without any error now, on my side

[fredrickflower]

Any fix for "Error loading script: trt.py" then?

[AugmentedRealityCat]

Here is what I did to get pycuda to work. I am not saying YOU should do that. I am sure there are steps that are not necessary, and some might even break things that are currently working. But I'll share my experience and you do what you want with it, at your own risk. This is all on Windows 10, using the latest DEV branch.

I had first installed the Visual Studio C++ build tools but it was version 2022, which is the one offered by default by Microsoft. It was possible to include all the packages required, it seems, but it was not enough to get the whole thing to work. So I removed Visual C++ Build Tools 2022.

After removing version 2022, I installed the version shown in the screenshot above, Visual C++ Build Tools 2019. I selected exactly the packages required, but I would still get errors about nvcc.exe and path and Cuda (if I remember correctly). But it was different and it looked like I was progressing.

I was about to change the path invormation linking to CUDA in the environment variables when I discovered that nvcc.exe and cuda were missing. I did a system restore yesterday to try to fix things, and maybe that's why it was gone. I don't know. Anyways, without it installed, I could not link to it in the environment variables. So I tried to find how to install nvcc.exe and cuda, which was to install the CUDA toolkit from Nvidia. And I remembered to check which version was compatible with Torch 2.0, so v 11.8.

I then installed that version of Cuda (which has nvcc.exe). It is important to note that CUDA was installed AFTER installing the Microsoft Visual C++ Build Tools 2019 with all the required packages. The version I installed was cuda_11.8.0_522.06_windows.

And only after that could I get the extension to install properly, and its tab to appear, and the whole thing to work without error messages.

Now, I managed to convert a safetensor into a onnx, and that onnx into a trt, and I got over 60 it/s when generating images with it after, while normally I get 30 it/s max in the best case scenario.

I hope my little story will be helpful for someone ! (but use at your own risk - I am NOT a programmer, far from it)

TLDR:

1. removed Visual C++ build tools 2022
2. installed Visual C++ build tools 2019 + required packages
3. installed Nvidia Cuda Toolkit v11.8 (which works with Torch 2)
4. installed the extension properly without errors after that

[reallybigname]

Verified. I had to update Visual Studio Build Tools before it would install pycuda and work. I'm using Build Tools 2022.

[darongma]

Here is what I did to get pycuda to work. I am not saying YOU should do that. I am sure there are steps that are not necessary, and some might even break things that are currently working. But I'll share my experience and you do what you want with it, at your own risk. This is all on Windows 10, using the latest DEV branch.

I had first installed the Visual Studio C++ build tools but it was version 2022, which is the one offered by default by Microsoft. It was possible to include all the packages required, it seems, but it was not enough to get the whole thing to work. So I removed Visual C++ Build Tools 2022.

After removing version 2022, I installed the version shown in the screenshot above, Visual C++ Build Tools 2019. I selected exactly the packages required, but I would still get errors about nvcc.exe and path and Cuda (if I remember correctly). But it was different and it looked like I was progressing.

I was about to change the path invormation linking to CUDA in the environment variables when I discovered that nvcc.exe and cuda were missing. I did a system restore yesterday to try to fix things, and maybe that's why it was gone. I don't know. Anyways, without it installed, I could not link to it in the environment variables. So I tried to find how to install nvcc.exe and cuda, which was to install the CUDA toolkit from Nvidia. And I remembered to check which version was compatible with Torch 2.0, so v 11.8.

I then installed that version of Cuda (which has nvcc.exe). It is important to note that CUDA was installed AFTER installing the Microsoft Visual C++ Build Tools 2019 with all the required packages. The version I installed was cuda_11.8.0_522.06_windows.

And only after that could I get the extension to install properly, and its tab to appear, and the whole thing to work without error messages.

Now, I managed to convert a safetensor into a onnx, and that onnx into a trt, and I got over 60 it/s when generating images with it after, while normally I get 30 it/s max in the best case scenario.

I hope my little story will be helpful for someone ! (but use at your own risk - I am NOT a programmer, far from it)

TLDR:

1. removed Visual C++ build tools 2022
2. installed Visual C++ build tools 2019 + required packages
3. installed Nvidia Cuda Toolkit v11.8 (which works with Torch 2)
4. installed the extension properly without errors after that

Thanks for this info. It helps me and I finally managed to get it working.

So, I follow direction to have the extension install on \stable-diffusion-webui\extensions\stable-diffusion-webui-tensorrt Then, I extract the nvidia stuff and put it into \stable-diffusion-webui\extensions\stable-diffusion-webui-tensorrt\TensorRT-8.6.1.6

Then, I launch webui-user.bat and it give me a bunch of errors about not able to install pycuda.

To fix all that, I just install the following, then everything works.

1. Install Visual C++ build tools 2022
2. Install Nvidia Cuda Toolkit v11.8

[cheadrian]

` WARNING: nvcc not in path.

May need to set CUDA_INC_DIR for installation to succeed.`

Installing Cuda Toolkit V11.8 fixed this issue.

Also, I have Visual C++ build tools 2022 and 2019 installed. Don't think both are necessary.

Windows 11 / 10 SDK should not be necessary.