Closed choosehappy closed 3 months ago
A quick follow up, with v2.4.0a0, it actually doesn't to that line, there is an error above in creating the trt_model:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In[17], line 6
4 input_tensor = images.cuda()
5 exp_program = torch.export.export(model, (input_tensor,))
----> 6 trt_model = torchtrt.dynamo.compile(
7 exp_program,
8 inputs=[input_tensor],
9 enabled_precisions={torch.float8_e4m3fn},
10 min_block_size=1,
11 debug=False,
12 )
14 # Inference compiled Torch-TensorRT model over the testing dataset
15 total = 0
--snip--
TypeError: Provided an unsupported data type as an input data type (support: bool, int32, long, half, float), got: torch.float8_e4m3fn
@zewenli98 can you take a look ?
Might be a copy paste error:
https://github.com/pytorch/TensorRT/blob/52ba6f1e3ff5b905e15a12b500af2d4abf847e21/examples/dynamo/vgg16_fp8_ptq.py#L239
above has this code:
and then a note saying
but then it looks like inference is done with the original model?