Closed tms2003 closed 2 years ago
stale[bot]
commented
2 years ago This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Lenan22
commented
2 years ago Env
- GPU,RTX2060
- OS, e.g. Ubuntu20.04 etc.
- Cuda 11.4 version
- TensorRT 8
About this repo
- which branch/tag/commit are you using?
- which model? yolov5,
Your problem
- what is your command? sudo ./yolov5 -s yolov5m6.wts yolov5m6_int8.engine m6
- what's your output?
[11/30/2021-21:56:28] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer 451) [ElementWise]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor [11/30/2021-21:56:28] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer 452) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor [11/30/2021-21:56:28] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer 453) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor [11/30/2021-21:56:28] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer 455) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor [11/30/2021-21:56:29] [E] [TRT] 2: [graphOptimizer.cpp::mergeLayers::6620] Error Code 2: Internal Error (Assertion nbPTQScales == nbChannels failed.) Build engine successfully! yolov5: /home/tms/source/tensorrtx/yolov5/yolov5.cpp:243: void APIToModel(unsigned int, nvinfer1::IHostMemory**, bool&, float&, float&, std::string&): Assertion `engine != nullptr' failed. 已放弃
- what output do you expect?
convert succefual!
I got succe with yolov5m6 fp16 and yolov5x6 int8 /yolov5x int8 .so ,maybe it's a bug
Please refer to our open source quantization tool ppq, the quantization result is better than the quantization tool that comes with tensorrt, almost the same as the float32 model. https://github.com/openppl-public/ppq/blob/master/md_doc/deploy_trt_by_api.md
Env
About this repo
Your problem
what is your command? sudo ./yolov5 -s yolov5m6.wts yolov5m6_int8.engine m6
what's your output?
[11/30/2021-21:56:28] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer 451) [ElementWise]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor [11/30/2021-21:56:28] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer 452) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor [11/30/2021-21:56:28] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer 453) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor [11/30/2021-21:56:28] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer 455) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor [11/30/2021-21:56:29] [E] [TRT] 2: [graphOptimizer.cpp::mergeLayers::6620] Error Code 2: Internal Error (Assertion nbPTQScales == nbChannels failed.) Build engine successfully! yolov5: /home/tms/source/tensorrtx/yolov5/yolov5.cpp:243: void APIToModel(unsigned int, nvinfer1::IHostMemory**, bool&, float&, float&, std::string&): Assertion `engine != nullptr' failed. 已放弃
convert succefual!
I got succe with yolov5m6 fp16 and yolov5x6 int8 /yolov5x int8 .so ,maybe it's a bug