mistralai/Mistral-7B-v0.3 is not fully supported because the index inputs to torch.ops.aten.index.Tensor are half-precision according to the fake tensor metadata.
The graph consists of 2895 Total Operators, of which 2831 operators are supported, 97.79% coverage
The following ops are currently unsupported or excluded from conversion, and are listed with their op-count in the graph:
torch.ops.aten.index.Tensor: 64
Environment
Build information about Torch-TensorRT can be found by turning on debug messages
Bug Description
mistralai/Mistral-7B-v0.3is not fully supported because the index inputs totorch.ops.aten.index.Tensorare half-precision according to the fake tensor metadata.To Reproduce
torch.compileand run inference on themistralai/Mistral-7B-v0.3modelExpected behavior
Environment
2.3.02.3.1+cu1214.40.2