Open LeoRainly opened 2 years ago
grimoire
commented
2 years ago The warning about myelin happened when the model is too complex and too many shape inference need to be performed. Try to avoid dynamic shape (you can set the same shape in opt_shape_param). TensorRt 7.1 is an old version and might have unexpected problems. Better update the TensorRT version.
LeoRainly
commented
2 years ago Thank you so much. By the way, I also got these warnings:
unknown interpolate type, use linear instead. Warning: Encountered known unsupported method torch.nn.functional.has_torch_function Warning: Encountered known unsupported method torch.equal Warning: Encountered known unsupported method torch.equal Warning: Encountered known unsupported method torch.nn.functional.multi_head_attention_forward
Does these warnings really matter? Do I need to do something to clear them? Thanks.
grimoire
commented
2 years ago It is hard to say, the warning means there is an unsupported layer in the network. The output of this node would be treated as constant value. So if these nodes can be replaced with a constant value of its output, you can just ignore the warning (for example, the anchor of a detection model). Or sometimes you have to do some modification of your model such as:
if a==b: # which might use `torch.equal`
c=d
else:
c=e
# assume `a==b` is constant value might bring some error
# can be replaced with
c=torch.where(a==b, d, e)I set the same shape in opt_shape_param, then _testmodel.plan and _testmodel.pth are generated (maybe in an incorrect way). However, when I step through the code of my network line by line with pdb, the following error appears:
[TensorRT] ERROR: (Unnamed Layer 5101) [Constant]: invalid weights type of Bool
[TensorRT] ERROR: (Unnamed Layer 5101) [Constant]: invalid weights type of Bool
[TensorRT] ERROR: Layer (Unnamed Layer* 5101) [Constant] failed validation
[TensorRT] ERROR: Network validation failed.
Traceback (most recent call last):
File "/mydata/mmdetection-to-tensorrt/tests/model_test.py", line 144, in
I don't know why the weights of some layers are of Bool type, how can I repair this error ? Thank you again~
Hi, the error I got:
[TensorRT] INFO: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output. [TensorRT] WARNING: Myelin graph with multiple dynamic values may have poor performance if they differ. Dynamic values are: (+ (FLOOR_DIV (FLOOR_DIV (FLOOR_DIV (FLOOR_DIV (FLOOR_DIV (+ (# 3 (SHAPE input_0)) -1) 2) 2) 2) 2) 2) 1) (+ (FLOOR_DIV (FLOOR_DIV (FLOOR_DIV (FLOOR_DIV (FLOOR_DIV (+ (# 2 (SHAPE input_0)) -1) 2) 2) 2) 2) 2) 1) [TensorRT] WARNING: Myelin graph with multiple dynamic values may have poor performance if they differ. Dynamic values are: (+ (FLOOR_DIV (FLOOR_DIV (FLOOR_DIV (FLOOR_DIV (FLOOR_DIV (+ (# 3 (SHAPE input_0)) -1) 2) 2) 2) 2) 2) 1) (+ (FLOOR_DIV (FLOOR_DIV (FLOOR_DIV (FLOOR_DIV (FLOOR_DIV (+ (# 2 (SHAPE input_0)) -1) 2) 2) 2) 2) 2) 1) [TensorRT] WARNING: Myelin graph with multiple dynamic values may have poor performance if they differ. Dynamic values are: (BROADCAST_SIZE (BROADCAST_SIZE (+ E0 1) (ONNX_RESIZE (+ E1 1) 2.000000e+00)) (ONNX_RESIZE (ONNX_RESIZE (+ (FLOOR_DIV E1 2) 1) 2.000000e+00) 2.000000e+00)) where E0=(FLOOR_DIV (FLOOR_DIV (FLOOR_DIV (+ (# 3 (SHAPE input_0)) -1) 2) 2) 2) E1=(FLOOR_DIV E0 2) (BROADCAST_SIZE (BROADCAST_SIZE (ONNX_RESIZE (+ E1 1) 2.000000e+00) (+ E0 1)) (ONNX_RESIZE (ONNX_RESIZE (+ (FLOOR_DIV E1 2) 1) 2.000000e+00) 2.000000e+00)) where E0=(FLOOR_DIV (FLOOR_DIV (FLOOR_DIV (+ (# 2 (SHAPE input_0)) -1) 2) 2) 2) E1=(FLOOR_DIV E0 2) [TensorRT] WARNING: Myelin graph with multiple dynamic values may have poor performance if they differ. Dynamic values are: (BROADCAST_SIZE (BROADCAST_SIZE (+ E0 1) (ONNX_RESIZE (+ E1 1) 2.000000e+00)) (ONNX_RESIZE (ONNX_RESIZE (+ (FLOOR_DIV E1 2) 1) 2.000000e+00) 2.000000e+00)) where E0=(FLOOR_DIV (FLOOR_DIV (FLOOR_DIV (+ (# 3 (SHAPE input_0)) -1) 2) 2) 2) E1=(FLOOR_DIV E0 2) (BROADCAST_SIZE (BROADCAST_SIZE (ONNX_RESIZE (+ E1 1) 2.000000e+00) (+ E0 1)) (ONNX_RESIZE (ONNX_RESIZE (+ (FLOOR_DIV E1 2) 1) 2.000000e+00) 2.000000e+00)) where E0=(FLOOR_DIV (FLOOR_DIV (FLOOR_DIV (+ (# 2 (SHAPE input_0)) -1) 2) 2) 2) E1=(FLOOR_DIV E0 2) [TensorRT] WARNING: Myelin graph with multiple dynamic values may have poor performance if they differ. Dynamic values are: (BROADCAST_SIZE (BROADCAST_SIZE (+ E0 1) (ONNX_RESIZE (+ E1 1) 2.000000e+00)) (ONNX_RESIZE (ONNX_RESIZE (+ (FLOOR_DIV E1 2) 1) 2.000000e+00) 2.000000e+00)) where E0=(FLOOR_DIV (FLOOR_DIV (FLOOR_DIV (+ (# 3 (SHAPE input_0)) -1) 2) 2) 2) E1=(FLOOR_DIV E0 2) (BROADCAST_SIZE (BROADCAST_SIZE (ONNX_RESIZE (+ E1 1) 2.000000e+00) (+ E0 1)) (ONNX_RESIZE (ONNX_RESIZE (+ (FLOOR_DIV E1 2) 1) 2.000000e+00) 2.000000e+00)) where E0=(FLOOR_DIV (FLOOR_DIV (FLOOR_DIV (+ (# 2 (SHAPE input_0)) -1) 2) 2) 2) E1=(FLOOR_DIV E0 2) [TensorRT] ERROR: ../builder/cudnnBuilderBlockChooser.cpp (117) - Assertion Error in buildMemGraph: 0 (mg.nodes[mg.regionIndices[outputRegion]].size == mg.nodes[mg.regionIndices[inputRegion]].size) Traceback (most recent call last): File "/mydata/mmdetection-to-tensorrt/tests/model_test.py", line 144, in
main()
File "/mydata/mmdetection-to-tensorrt/tests/model_test.py", line 131, in main
fp16=args.fp16)
File "/mydata/mmdetection-to-tensorrt/tests/model_test.py", line 41, in convert_test
torch.save(trt_model.state_dict(), trt_model_path)
File "/mydata/miniconda3/envs/mmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 909, in state_dict
hook_result = hook(self, destination, prefix, local_metadata)
File "/mydata/torch2trt_dynamic/torch2trt_dynamic/torch2trt_dynamic.py", line 467, in _on_state_dict
state_dict[prefix + 'engine'] = bytearray(self.engine.serialize())
AttributeError: 'NoneType' object has no attribute 'serialize'
I also had this problem with TensorRT 7.1.3.4
The WARNING "Myelin graph...." looks useful, however I can't understand its meaning.
Any advice is valuable for me. Thanks a lot.