Open kacper-kleczewski opened 3 weeks ago
Hello @kacper-kleczewski , I tried your script with the main branch (which is on 2.5.0.dev20240822+cu124
) and it works fine. Here's the slightly modified script that I tried
import torch
import torch_tensorrt
model = torch.nn.Linear(5, 7).eval().cuda()
sample = torch.randn(3, 5).cuda()
pyt_out = model(sample)
ep = torch.export.export(model, (sample,))
torch.export.save(ep, "model.ep")
ep_loaded = torch.export.load("model.ep")
compiled = torch_tensorrt.dynamo.compile(ep_loaded, [sample], min_block_size=1)
torch_tensorrt.save(compiled, "model_compiled.ep", inputs=[sample])
loaded_torch_tensorrt = torch.export.load("model_compiled.ep")
trt_gm = loaded_torch_tensorrt.module()
trt_out = trt_gm(sample)
print("Diff: ", torch.mean(torch.abs(pyt_out-trt_out)))
I remember we had some serialization issues with 2.4 version of torch which were resolved recently.
Bug Description
Models exported with
torch.export.export
, saved, loaded and then compiled withtorch_tensorrt.dynamo.compile
cannot be loaded withtorch.export.load
with error:To Reproduce
Code below should reproduce issue. It can be also observed with more complex model like EfficientNet.
Expected behavior
Succesfull loading of the model.
Environment
Nvidia PyTorch container 24.07