search

wang-xinyu / tensorrtx

Implementation of popular deep learning networks with TensorRT network definition API
MIT License
6.93k stars 1.77k forks source link

retinaface res50 int8 produce bad result #1093

Closed hungtooc closed 2 years ago

hungtooc commented 2 years ago

Env

About this repo

Your problem

wang-xinyu commented 2 years ago

This is normal. Int8 has bad result.

hungtooc commented 2 years ago

Thanks. The image in README made me confused

taotaoorange commented 2 years ago

@wang-xinyu But all those "Missing scale and zero-point for tensor (Unnamed Layer* 178)" indicate something is wrong, right? I only see these warnings for TensorRT 8.4

wang-xinyu commented 2 years ago

Yes, maybe some layers don't support int8.

Lenan22 commented 1 year ago

Env

  • GPU: Tesla T4
  • OS: Ubuntu 20.4
  • Cuda version 11.5
  • TensorRT version 8.4.0.6

About this repo

  • which branch main
  • which model retinaface

Your problem

  • what is your command:
sudo ./retina_r50 -s  // build and serialize model to file i.e. 'retina_r50.engine'
wget https://github.com/Tencent/FaceDetection-DSFD/raw/master/data/worlds-largest-selfie.jpg
sudo ./retina_r50 -d  // deserialize model file and run inference.
  • what's your output:
$ ./retina_r50 -s (I got error if run with sudo but work fine without it)
Loading weights: ../retinaface.wts
Your platform support int8: 1
Building engine, please wait for a while...
reading calib cache: r50_int8calib.table
reading calib cache: r50_int8calib.table
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 0) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 4) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 7) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 10) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 12) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 16) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 19) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 22) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 26) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 29) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 32) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 36) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 39) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 42) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 44) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 48) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 51) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 54) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 58) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 61) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 64) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 68) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 71) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 74) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 78) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 81) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 84) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 86) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 90) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 93) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 96) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 100) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 103) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 106) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 110) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 113) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 116) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 120) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 123) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 126) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 130) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 133) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 136) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 140) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 143) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 146) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 148) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 152) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 155) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 158) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 162) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 165) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 168) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 172) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 175) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 178) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 183) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 188) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 191) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 193) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 196) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 198) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 201) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 205) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 207) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 210) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 212) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 215) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 219) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 221) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 224) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 226) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 229) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:33] [W] [TRT] TensorRT was linked against cuBLAS/cuBLAS LT 11.8.0 but loaded cuBLAS/cuBLAS LT 11.7.4
[09/02/2022-08:34:33] [W] [TRT] TensorRT was linked against cuDNN 8.3.2 but loaded cuDNN 8.3.1
[09/02/2022-08:36:01] [W] [TRT] TensorRT was linked against cuBLAS/cuBLAS LT 11.8.0 but loaded cuBLAS/cuBLAS LT 11.7.4
[09/02/2022-08:36:01] [W] [TRT] TensorRT was linked against cuDNN 8.3.2 but loaded cuDNN 8.3.1
Build engine successfully

The result is bad, i've tried other images, which bad too: 0_result

  • what output do you expect?

Env

  • GPU: Tesla T4
  • OS: Ubuntu 20.4
  • Cuda version 11.5
  • TensorRT version 8.4.0.6

About this repo

  • which branch main
  • which model retinaface

Your problem

  • what is your command:
sudo ./retina_r50 -s  // build and serialize model to file i.e. 'retina_r50.engine'
wget https://github.com/Tencent/FaceDetection-DSFD/raw/master/data/worlds-largest-selfie.jpg
sudo ./retina_r50 -d  // deserialize model file and run inference.
  • what's your output:
$ ./retina_r50 -s (I got error if run with sudo but work fine without it)
Loading weights: ../retinaface.wts
Your platform support int8: 1
Building engine, please wait for a while...
reading calib cache: r50_int8calib.table
reading calib cache: r50_int8calib.table
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 0) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 4) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 7) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 10) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 12) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 16) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 19) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 22) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 26) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 29) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 32) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 36) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 39) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 42) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 44) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 48) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 51) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 54) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 58) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 61) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 64) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 68) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 71) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 74) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 78) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 81) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 84) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 86) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 90) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 93) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 96) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 100) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 103) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 106) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 110) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 113) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 116) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 120) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 123) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 126) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 130) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 133) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 136) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 140) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 143) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 146) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 148) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 152) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 155) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 158) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 162) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 165) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 168) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 172) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 175) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 178) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 183) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 188) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 191) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 193) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 196) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 198) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 201) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 205) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 207) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 210) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 212) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 215) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 219) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 221) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 224) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 226) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:31] [W] [TRT] Missing scale and zero-point for tensor (Unnamed Layer* 229) [Convolution]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
[09/02/2022-08:34:33] [W] [TRT] TensorRT was linked against cuBLAS/cuBLAS LT 11.8.0 but loaded cuBLAS/cuBLAS LT 11.7.4
[09/02/2022-08:34:33] [W] [TRT] TensorRT was linked against cuDNN 8.3.2 but loaded cuDNN 8.3.1
[09/02/2022-08:36:01] [W] [TRT] TensorRT was linked against cuBLAS/cuBLAS LT 11.8.0 but loaded cuBLAS/cuBLAS LT 11.7.4
[09/02/2022-08:36:01] [W] [TRT] TensorRT was linked against cuDNN 8.3.2 but loaded cuDNN 8.3.1
Build engine successfully

The result is bad, i've tried other images, which bad too: 0_result

  • what output do you expect?

Please refer to our open source quantization tool ppq, the quantization result is better than the quantization tool that comes with tensorrt, almost the same as the float32 model. https://github.com/openppl-public/ppq/blob/master/md_doc/deploy_trt_by_api.md