search

jkjung-avt / tensorrt_demos

TensorRT MODNet, YOLOv4, YOLOv3, SSD, MTCNN, and GoogLeNet
https://jkjung-avt.github.io/
MIT License
1.74k stars 545 forks source link

Xavier yolo to Int8 met error #580

Open herschelpuu opened 1 year ago

herschelpuu commented 1 year ago

Hi I've met the error below during converting from onnx model to int8 trt engine on Xavier

I just followed the step in instructions and exec: python3 onnx_to_tensorrt.py -v --int8 -m yolov4-int8-416 but got the below error when the code finished calibration and start to build the engine at the first layer:

[09/14/2022-14:46:27] [TRT] [V] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: -2027588946874785071 [09/14/2022-14:46:27] [TRT] [V] Autotuning format combination: Half(173056,1:4,416,1) -> Half(692224,1:8,1664,4) [09/14/2022-14:46:27] [TRT] [V] --------------- Timing Runner: 001_convolutional + 001_convolutional_bn (CaskConvolution) [09/14/2022-14:46:27] [TRT] [V] CaskConvolution has no valid tactics for this config, skipping [09/14/2022-14:46:27] [TRT] [V] Autotuning format combination: Half(173056,1:8,416,1) -> Float(5537792,173056,416,1) [09/14/2022-14:46:27] [TRT] [V] --------------- Timing Runner: 001_convolutional + 001_convolutional_bn (CudnnConvolution) [09/14/2022-14:46:27] [TRT] [V] CudnnConvolution has no valid tactics for this config, skipping [09/14/2022-14:46:27] [TRT] [V] --------------- Timing Runner: 001_convolutional + 001_convolutional_bn (CaskConvolution) [09/14/2022-14:46:27] [TRT] [V] CaskConvolution has no valid tactics for this config, skipping [09/14/2022-14:46:27] [TRT] [V] Autotuning format combination: Half(173056,1:8,416,1) -> Half(692224,1:8,1664,4) [09/14/2022-14:46:27] [TRT] [V] --------------- Timing Runner: 001_convolutional + 001_convolutional_bn (CudaDepthwiseConvolution) [09/14/2022-14:46:27] [TRT] [V] CudaDepthwiseConvolution has no valid tactics for this config, skipping [09/14/2022-14:46:27] [TRT] [V] --------------- Timing Runner: 001_convolutional + 001_convolutional_bn (CudnnConvolution) [09/14/2022-14:46:27] [TRT] [V] CudnnConvolution has no valid tactics for this config, skipping [09/14/2022-14:46:27] [TRT] [V] --------------- Timing Runner: 001_convolutional + 001_convolutional_bn (CaskConvolution) [09/14/2022-14:46:27] [TRT] [V] 001_convolutional + 001_convolutional_bn Set Tactic Name: sm70_xmma_fprop_implicit_gemm_f16f16_f16f16_f16_nhwckrsc_nhwc_tilesize128x256x32_stage1_warpsize2x4x1_g1_tensor8x8x4_t1r3s3 Tactic: 46202665595848747 [09/14/2022-14:46:27] [TRT] [V] Deleting timing cache: 3956 entries, 25642 hits [09/14/2022-14:46:27] [TRT] [E] 1: Unexpected exception ERROR: failed to build the TensorRT engine!

and just stopped. I just got the calib bin but no engine

my jetpack version is 4.6.1, onnx ==1.9.0, protobuf 3.8.0, TensorRT == 8.2.1.1, cuda ==10.2

I can use the calib_x.bin and onnx to generate trt engine by using /trtexec successfully

/txtexec --onnx=xxxx --calib=xxxx --saveEngine=xxx --verbose --int8

(but without yolo_layer. I havent read any materials about how to plugin this way). I can convert onnx to trt in FP16 mode and use trt_yolo.py to detect without any problem. I've also tested commenting lines about "plugins" in onnx_to_tensorrt.py to see if it is because of the plugins:

network = add_yolo_plugins(network, model_name, TRT_LOGGER)

network = add_concat(network, model_name, TRT_LOGGER)

But it didn't work neither, and reported the same error as above. I guess it't not because of the plugins, but tensorrt API? ps I also tested the demo on jetpack 5.0 version. onnx_to_tensorrt.py doesn't work neither. But it reported as "CudaError"

Do you have any experience about this ?

Much appreciation!

jkjung-avt commented 1 year ago

I have not seen this error before. Which version of TensorRT are you using?

[09/14/2022-14:46:27] [TRT] [V] Deleting timing cache: 3956 entries, 25642 hits [09/14/2022-14:46:27] [TRT] [E] 1: Unexpected exception ERROR: failed to build the TensorRT engine!

Lenan22 commented 1 year ago

Hi I've met the error below during converting from onnx model to int8 trt engine on Xavier

I just followed the step in instructions and exec: python3 onnx_to_tensorrt.py -v --int8 -m yolov4-int8-416 but got the below error when the code finished calibration and start to build the engine at the first layer:

[09/14/2022-14:46:27] [TRT] [V] >>>>>>>>>>>>>>> Chose Runner Type: CaskConvolution Tactic: -2027588946874785071 [09/14/2022-14:46:27] [TRT] [V] Autotuning format combination: Half(173056,1:4,416,1) -> Half(692224,1:8,1664,4) [09/14/2022-14:46:27] [TRT] [V] --------------- Timing Runner: 001_convolutional + 001_convolutional_bn (CaskConvolution) [09/14/2022-14:46:27] [TRT] [V] CaskConvolution has no valid tactics for this config, skipping [09/14/2022-14:46:27] [TRT] [V] Autotuning format combination: Half(173056,1:8,416,1) -> Float(5537792,173056,416,1) [09/14/2022-14:46:27] [TRT] [V] --------------- Timing Runner: 001_convolutional + 001_convolutional_bn (CudnnConvolution) [09/14/2022-14:46:27] [TRT] [V] CudnnConvolution has no valid tactics for this config, skipping [09/14/2022-14:46:27] [TRT] [V] --------------- Timing Runner: 001_convolutional + 001_convolutional_bn (CaskConvolution) [09/14/2022-14:46:27] [TRT] [V] CaskConvolution has no valid tactics for this config, skipping [09/14/2022-14:46:27] [TRT] [V] Autotuning format combination: Half(173056,1:8,416,1) -> Half(692224,1:8,1664,4) [09/14/2022-14:46:27] [TRT] [V] --------------- Timing Runner: 001_convolutional + 001_convolutional_bn (CudaDepthwiseConvolution) [09/14/2022-14:46:27] [TRT] [V] CudaDepthwiseConvolution has no valid tactics for this config, skipping [09/14/2022-14:46:27] [TRT] [V] --------------- Timing Runner: 001_convolutional + 001_convolutional_bn (CudnnConvolution) [09/14/2022-14:46:27] [TRT] [V] CudnnConvolution has no valid tactics for this config, skipping [09/14/2022-14:46:27] [TRT] [V] --------------- Timing Runner: 001_convolutional + 001_convolutional_bn (CaskConvolution) [09/14/2022-14:46:27] [TRT] [V] 001_convolutional + 001_convolutional_bn Set Tactic Name: sm70_xmma_fprop_implicit_gemm_f16f16_f16f16_f16_nhwckrsc_nhwc_tilesize128x256x32_stage1_warpsize2x4x1_g1_tensor8x8x4_t1r3s3 Tactic: 46202665595848747 [09/14/2022-14:46:27] [TRT] [V] Deleting timing cache: 3956 entries, 25642 hits [09/14/2022-14:46:27] [TRT] [E] 1: Unexpected exception ERROR: failed to build the TensorRT engine!

and just stopped. I just got the calib bin but no engine

my jetpack version is 4.6.1, onnx ==1.9.0, protobuf 3.8.0, TensorRT == 8.2.1.1, cuda ==10.2

I can use the calib_x.bin and onnx to generate trt engine by using /trtexec successfully

/txtexec --onnx=xxxx --calib=xxxx --saveEngine=xxx --verbose --int8

(but without yolo_layer. I havent read any materials about how to plugin this way). I can convert onnx to trt in FP16 mode and use trt_yolo.py to detect without any problem. I've also tested commenting lines about "plugins" in onnx_to_tensorrt.py to see if it is because of the plugins:

network = add_yolo_plugins(network, model_name, TRT_LOGGER) #network = add_concat(network, model_name, TRT_LOGGER)

But it didn't work neither, and reported the same error as above. I guess it't not because of the plugins, but tensorrt API? ps I also tested the demo on jetpack 5.0 version. onnx_to_tensorrt.py doesn't work neither. But it reported as "CudaError"

Do you have any experience about this ?

Much appreciation! Please refer to our open source quantization tool ppq, we can help you solve quantization problems https://github.com/openppl-public/ppq/blob/master/md_doc/deploy_trt_by_OnnxParser.md