narendasan
commented
2 weeks ago Seems like an issue related to TensorRT https://github.com/NVIDIA/TensorRT/issues/3983. I would recommend using TensorRT 10.0 with Torch-TensorRT as that is the current validated version
minimok7
Bug Description
I got this error concerning "libnvinfer_builder_resource_win.so.10.2.0" I couldn't find this file. I think this is related to the window env but my env is ubuntu.
WARNING:torch_tensorrt.dynamo._compiler:Node _param_constant0 of op type get_attr does not have metadata. This could sometimes lead to undefined behavior. WARNING:torch_tensorrt.dynamo._compiler:Some nodes do not have metadata (shape and dtype information). This could lead to problems sometimes if the graph has PyTorch and TensorRT segments. INFO:torch_tensorrt [TensorRT Conversion Context]:[MemUsageChange] Init CUDA: CPU +2, GPU +0, now: CPU 279, GPU 612 (MiB) INFO:torch_tensorrt [TensorRT Conversion Context]:[MemUsageChange] Init builder kernel library: CPU +2170, GPU +414, now: CPU 2602, GPU 1026 (MiB) INFO:torch_tensorrt.dynamo.conversion._TRTInterpreter:TRT INetwork construction elapsed time: 0:00:00.008743 INFO:torch_tensorrt [TensorRT Conversion Context]:Global timing cache in use. Profiling results in this builder pass will be stored. ERROR:torch_tensorrt [TensorRT Conversion Context]:IBuilder::buildSerializedNetwork: Error Code 6: API Usage Error (Unable to load library: libnvinfer_builder_resource_win.so.10.2.0: libnvinfer_builder_resource_win.so.10.2.0: cannot open shared object file: No such file or directory) WARNING:torch_tensorrt.dynamo.backend.backends:TRT conversion failed on the subgraph. See trace above. Returning GraphModule forward instead.
To Reproduce
import torch import torch_tensorrt import torch.nn as nn
torch.manual_seed(0) torch.cuda.manual_seed(0) device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = nn.Sequential(nn.Conv2d(3,128,kernel_size=3,padding=1), nn.ReLU(), nn.Conv2d(128,256,kernel_size=3,padding=1, stride=2), nn.ReLU(), nn.Conv2d(256, 256, kernel_size=3, padding=1, stride=1), nn.ReLU(), nn.Conv2d(256, 256, kernel_size=3, padding=1, stride=1), nn.ReLU(), nn.Conv2d(256, 256, kernel_size=3, padding=1, stride=1), nn.ReLU(), nn.Conv2d(256, 256, kernel_size=3, padding=1, stride=1), nn.ReLU(), nn.Conv2d(256,2,kernel_size=3,padding=1, stride=1)) model = model.to(device).eval()
x = torch.rand(1,3,128,128).float()
inputs=[x.to("cuda")]
model = model.eval().to("cuda") exp_program = torch.export.export(model, tuple(inputs)) trt_gm = torch_tensorrt.dynamo.compile(exp_program, inputs) # Output is a torch.fx.GraphModule trt_gm(*inputs)
Expected behavior
I expected the result in this tutorial. https://pytorch.org/TensorRT/dynamo/dynamo_export.html
Environment
Additional context