Closed mlcoop closed 5 years ago
Did you follow the PyTorch instructions to export to a .onnx file? https://pytorch.org/docs/stable/onnx.html
It appears that you did as I see you may have generated text_clf_opset7.onnx
Once you have the .onnx model, you can use trtexec
found inside the NGC TensorRT container to build a TRT engine file. There is an example here: https://github.com/NVIDIA/tensorrt-laboratory/blob/master/examples/ONNX/resnet50/build.py
try this from the directory that contains your .onnx file, replacing name-of-your.onnx
ls *.onnx
nvidia-docker run --rm -ti -v $PWD:/work --workdir /work nvcr.io/nvidia/tensorrt:19.04-py3 bash
ls *.onnx
trtexec --onnx=name-of-your.onnx --batch=1 --saveEngine=test.engine
@ryanolson hey!
Unsupported ONNX data type: INT64 (7) [E] [TRT] Parameter check failed at: ../builder/Layers.cpp::ConstantLayer::1585, condition: weights.type == DataType::kFLOAT || weights.type == DataType::kHALF || weights.type == DataType::kINT32 While parsing node number 17 [Gather]: ERROR: onnx2trt_utils.hpp:275 In function convert_axis: [8] Assertion failed: axis >= 0 && axis < nbDims [E] failed to parse onnx file [E] Engine could not be created [E] Engine could not be created &&&& FAILED TensorRT.trtexec # trtexec --onnx=watermark.onnx --batch=1 --saveEngine=test.engine`
Shoul I rewrite model in Pytorch 1.0 ?
Let's tackle the first error:
Unsupported ONNX data type: INT64 (7)
Is there anywhere in the model definition that you are explicitly using python int
or using np.int64
? If you have python list/array and convert it to numpy, it will be an np.int64
value.
>>> import numpy as np
>>> a = [1]
>>> b = np.asarray(a)
>>> b
array([1])
>>> b.dtype
dtype('int64')
Try to weed out any integer values that might be set to np.int64
since TensorRT only support INT32 and INT8.
@ryanolson , The model is pretty simple, without any numpy in it. Only nn.Module from PyTorch.
TensorRT doesn't support np.int64
, so we need to determine which bits are using np.int64
and convert them to np.int32
So I converted all layers into torch.FloatTensor then did
root@33aa966b729c:/work# trtexec --model='watermark_float.onnx' --batch=1 --saveEngine='test.engine' &&&& RUNNING TensorRT.trtexec # trtexec --model=watermark_float.onnx --batch=1 --saveEngine=test.engine [I] model: watermark_float.onnx [I] batch: 1 [I] saveEngine: test.engine [E] Deploy file not specified [E] Engine could not be created &&&& FAILED TensorRT.trtexec # trtexec --model=watermark_float.onnx --batch=1 --saveEngine=test.engine
It fails with "Deploy file not specified"
Change --model
to --onnx
Please open if you need more assistance.
I have models built in PyTorch and also converted into onnx. PyTorch 1.0 TensortRt 51.5.0
First, tried to build engine as it was built in mnist example. TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
class ModelDataWatermark(object): INPUT_NAME = "data" INPUT_SHAPE = (1, 128, 128) OUTPUT_NAME = "prob" OUTPUT_SIZE = 2 DTYPE = trt.float32
` def populate_network_watermark(network, weights):
checkpoints = '/home/malibayev/kolesa-image-moderation-sanic/static/models/torch_model_07_06.pt' model = WatermarkModel(num_classes=2) model.load_state_dict(torch.load(checkpoints)) weights = model.state_dict()
`def build_engine(weights):
For more information on TRT basics, refer to the introductory samples.
engine = build_engine(weights)
AttributeError: enter was given
Then I tried to use Onnx Parser.
` class ModelData(object): MODEL_PATH = "/home/malibayev/kolesa-image-moderation-sanic/text_clf_opset7.onnx" INPUT_SHAPE = (3, 224, 224)
We can convert TensorRT data types to numpy types with trt.nptype()
builder = trt.Builder(TRT_LOGGER) network = builder.create_network() parser = trt.OnnxParser(network, TRT_LOGGER) builder.max_workspace_size = common.GiB(1) model = open('/home/malibayev/kolesa-image-moderation-sanic/text_clf_opset7.onnx', 'rb') parser.parse(model.read()) engine = builder.build_cuda_engine(network)
which also returned None parser.parse(model.read()) returns FalsePlease Help