search

NVIDIA / trt-samples-for-hackathon-cn

Simple samples for TensorRT programming
Apache License 2.0
1.5k stars 338 forks source link

TensorRT8.6 C++量化bug修改 #80

Open FelixFu520 opened 1 year ago

FelixFu520 commented 1 year ago

1，在createCalibrationAndInferenceData.py文件中，保存的npz数据是float64的，修改为float32 2，在calibrator.cpp文件中，有bufferSize为0时作为除数的bug，还有pData未分配内存空间，导致量化失败

FelixFu520 commented 1 year ago

有问题的代码：https://github.com/NVIDIA/trt-samples-for-hackathon-cn/blob/07271c3123b997b69842274590bf4cca017d6cb8/cookbook/04-BuildEngineByONNXParser/pyTorch-ONNX-TensorRT/C%2B%2B/calibrator.cpp#L10C4-L10C50

79