Can please somebody give a clear explanation of how to install torch-tensorrt on Windows?

[ninono12345]

❓ Question

Hello,

I've encountered problems installing torch-tensorrt on Windows 10

No matter how I try, how many sources I look up to, there is no clear explanation on how to do everything. The documentation is vague, and because I am used to working with python code, which does everything for you, that is pip install... python code.py, and nothing more is required, I do not have as much experience with cmake, building libraries, files, and c++, which makes it very difficult to follow along the installation process.

Now I've tried to follow along instructions from the main page

pip install torch-tensorrt doesn't work downloaded zip file of this repository; python setup.py install also doesn't work

installed bazel modified the workspace, still nothing

tried to directly import into code py/torch-tensorrt - nothing

then inside the py folder opened command prompt ant typed in:

bazel build //:libtorchtrt --compilation_mode=dbg

and received this error:

`Starting local Bazel server and connecting to it... INFO: Repository libtorch instantiated at: D:/pyth/tensorrt-main/WORKSPACE:53:13: in Repository rule http_archive defined at: C:/users/tomas/_bazel_tomas/r4zfvyvs/external/bazel_tools/tools/build_defs/repo/http.bzl:372:31: in WARNING: Download from https://download.pytorch.org/libtorch/nightly/cu121/libtorch-cxx11-abi-shared-with-deps-latest.zip failed: class com.google.devtools.build.lib.bazel.repository.downloader.ContentLengthMismatchException Bytes read 2210658461 but wanted 2501377827 ERROR: An error occurred during the fetch of repository 'libtorch': Traceback (most recent call last): File "C:/users/tomas/_bazel_tomas/r4zfvyvs/external/bazel_tools/tools/build_defs/repo/http.bzl", line 132, column 45, in _http_archive_impl download_info = ctx.download_and_extract( Error in download_and_extract: java.io.IOException: Error downloading [https://download.pytorch.org/libtorch/nightly/cu121/libtorch-cxx11-abi-shared-with-deps-latest.zip] to C:/users/tomas/_bazel_tomas/r4zfvyvs/external/libtorch/temp7217651597570855917/libtorch-cxx11-abi-shared-with-deps-latest.zip: Bytes read 2210658461 but wanted 2501377827 ERROR: D:/pyth/tensorrt-main/WORKSPACE:53:13: fetching http_archive rule //external:libtorch: Traceback (most recent call last): File "C:/users/tomas/_bazel_tomas/r4zfvyvs/external/bazel_tools/tools/build_defs/repo/http.bzl", line 132, column 45, in _http_archive_impl download_info = ctx.download_and_extract( Error in download_and_extract: java.io.IOException: Error downloading [https://download.pytorch.org/libtorch/nightly/cu121/libtorch-cxx11-abi-shared-with-deps-latest.zip] to C:/users/tomas/_bazel_tomas/r4zfvyvs/external/libtorch/temp7217651597570855917/libtorch-cxx11-abi-shared-with-deps-latest.zip: Bytes read 2210658461 but wanted 2501377827 ERROR: D:/pyth/tensorrt-main/core/util/logging/BUILD:13:11: //core/util/logging:logging depends on @libtorch//:libtorch in repository @libtorch which failed to fetch. no such package '@libtorch//': java.io.IOException: Error downloading [https://download.pytorch.org/libtorch/nightly/cu121/libtorch-cxx11-abi-shared-with-deps-latest.zip] to C:/users/tomas/_bazel_tomas/r4zfvyvs/external/libtorch/temp7217651597570855917/libtorch-cxx11-abi-shared-with-deps-latest.zip: Bytes read 2210658461 but wanted 2501377827 ERROR: Analysis of target '//:libtorchtrt' failed; build aborted: INFO: Elapsed time: 458.697s INFO: 0 processes. FAILED: Build did NOT complete successfully (64 packages loaded, 413 targets configured) Fetching https://download.pytorch.org/...orch-cxx11-abi-shared-with-deps-latest.zip; 2.1 GiB (2,210,121,825B) 446s

And also tried some other things, I cannot remember, but unsuccessfully.

THANK YOU FOR YOUR HELP IN ADVANCE

Environment

Build information about Torch-TensorRT can be found by turning on debug messages

• PyTorch Version: 2.1.2+cu121
• OS : Windows 10
• I am running python and pytorch straight from Windows, without any environment
• Python version: 3.10.13
• CUDA version: 12.1 update 1
• GPU models and configuration: GTX 1660 TI

[jensdraht1999]

Okay thanks, I will try as fast as possible. I will report anything to you.

[jensdraht1999]

@gs-olive I did work without a error. If you want to read the log file anyway, you can. Thank you very much. 1.log

[jensdraht1999]

@gs-olive I wanted to ask, if you intent to implement that we can cache tensorrt to a file, so it has not to calculate everything?

[gs-olive]

If you are referring to model export, #2806 should enable this workflow via the C++ runtime.

[jensdraht1999]

If you are referring to model export, #2806 should enable this workflow via the C++ runtime.

@gs-olive

I mean the model export on Pytorch.

You know, that there are 3 kinds of pytorch workflows: Normal, compiled, tensorrt.

Compiled and tensorrt are not available in pytorch ON WINDOWS since forever. Then a few days before this question was asked, there was pytorch compiled in a PR, however it performed very bad, so as far as I know, it will not be added.

TensorRT from Nvidia has been supported with you help, but the model, which will be optimized and then used for inferencing has not been generated like it does on Linux.

Let's say, I want to use "RESNET50" for inferencing. Let's say my performance is 50 per second. The I use TensorRT of NVIDIA, this will optimize the model and save the optimized model to a directory I like and the name of the file will be for example "resnet-rtx3060.onnx". After that it will use this model and this model will only be compatible with my gpu. And because it's compiled/optimized for my NVIDIA GPU it's 75 per second and probably even more. The range of how it good it performs is aroud 30%-200%.

This is the example code I mean (https://pytorch.org/TensorRT/tutorials/getting_started_with_fx_path.html):

def compile( module: nn.Module, input, max_batch_size=2048, max_workspace_size=33554432, explicit_batch_dimension=False, lower_precision=LowerPrecision.FP16, verbose_log=False, timing_cache_prefix="", save_timing_cache=False, cuda_graph_batch_size=-1, dynamic_batch=True, ) -> nn.Module:

I'm 100% honest with you, I do not know a lot, perhaps this PR is something else. I apologize if this something totally unrealeaded.