search

marcoslucianops / DeepStream-Yolo

NVIDIA DeepStream SDK 7.0 / 6.4 / 6.3 / 6.2 / 6.1.1 / 6.1 / 6.0.1 / 6.0 / 5.1 implementation for YOLO models
MIT License
1.38k stars 343 forks source link

Low frame rate YOLOv8, calibration int8 #500

Open Halacky opened 6 months ago

Halacky commented 6 months ago

My GPU GPU GTX 1070

I developing on Docker The versions I'm working with

deepstream-app version 6.3.0
DeepStreamSDK 6.3.0
CUDA Driver Version: 12.1
CUDA Runtime Version: 12.1
TensorRT Version: 8.5
cuDNN Version: 8.9
libNVWarp360 Version: 2.0.1d3

I'm running this

git clone https://github.com/ultralytics/ultralytics.git
cd ultralytics
pip3 install -r requirements.txt
python3 setup.py install
pip3 install onnx onnxsim onnxruntime

Then python3 export_yoloV8.py -w testbest.pt --dynamic

then Copy the generated ONNX model file and labels.txt file (if generated) to the DeepStream-Yolo folder.

I'm launching it CUDA_VER=12.1 make -C nvdsinfer_custom_impl_Yolo Already here I do not understand where the file model-engine-file=model_b1_gpu0_int8.engine should come from

So all I got was the onnx model and labels.txt

I decided to change config_infer_primary_yoloV8.txt as follows

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
model-color-format=0
onnx-file=/opt/nvidia/deepstream/deepstream-6.3/sources/deepstream_python_apps/apps/deepstream-test3/calib/DeepStream-Yolo/testbest.onnx
model-engine-file=model_b1_gpu0_int8.engine
int8-calib-file=calib.table
labelfile-path=labels.txt
batch-size=1
network-mode=1
num-detected-classes=3
interval=0
gie-unique-id=1
process-mode=1
network-type=0
cluster-mode=2
maintain-aspect-ratio=1
symmetric-padding=1
#workspace-size=2000
parse-bbox-func-name=NvDsInferParseYolo
#parse-bbox-func-name=NvDsInferParseYoloCuda
custom-lib-path=/opt/nvidia/deepstream/deepstream-6.3/sources/deepstream_python_apps/apps/deepstream-test3/calib/DeepStream-Yolo/nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so
engine-create-func-name=NvDsInferYoloCudaEngineGet

[class-attrs-all]
nms-iou-threshold=0.45
pre-cluster-threshold=0.25
topk=300

and I get the following set of errors

WARNING: [TRT]: CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars
WARNING: ../nvdsinfer/nvdsinfer_model_builder.cpp:659 INT8 calibration file not specified/accessible. INT8 calibration can be done through setDynamicRange API in 'NvDsInferCreateNetwork' implementation
WARNING: [TRT]: onnx2trt_utils.cpp:377: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
WARNING: [TRT]: onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
WARNING: [TRT]: Tensor DataType is determined at build time for tensors not marked as input or output.

Building the TensorRT Engine

File does not exist: /opt/nvidia/deepstream/deepstream-6.3/sources/deepstream_python_apps/apps/deepstream-test3/calib/DeepStream-Yolo/calib.table
OpenCV is required to run INT8 calibrator

Failed to build CUDA engine
ERROR: ../nvdsinfer/nvdsinfer_model_builder.cpp:728 Failed to create network using custom network creation function
ERROR: ../nvdsinfer/nvdsinfer_model_builder.cpp:794 Failed to get cuda engine from custom library API
0:00:04.926481923 141577 0x55e8c64cf2c0 ERROR                nvinfer gstnvinfer.cpp:676:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:2022> [UID = 1]: build engine file failed
0:00:04.937754138 141577 0x55e8c64cf2c0 ERROR                nvinfer gstnvinfer.cpp:676:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2108> [UID = 1]: build backend context failed
0:00:04.937771198 141577 0x55e8c64cf2c0 ERROR                nvinfer gstnvinfer.cpp:676:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1282> [UID = 1]: generate backend failed, check config file settings
0:00:05.107022653 141577 0x55e8c64cf2c0 WARN                 nvinfer gstnvinfer.cpp:898:gst_nvinfer_start:<primary_gie> error: Failed to create NvDsInferContext instance
0:00:05.107039197 141577 0x55e8c64cf2c0 WARN                 nvinfer gstnvinfer.cpp:898:gst_nvinfer_start:<primary_gie> error: Config file path: /opt/nvidia/deepstream/deepstream-6.3/sources/deepstream_python_apps/apps/deepstream-test3/calib/DeepStream-Yolo/config_infer_primary_yoloV8.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED
** ERROR: <main:716>: Failed to set pipeline to PAUSED
Quitting
nvstreammux: Successfully handled EOS for source_id=0
ERROR from primary_gie: Failed to create NvDsInferContext instance
Debug info: gstnvinfer.cpp(898): gst_nvinfer_start (): /GstPipeline:pipeline/GstBin:primary_gie_bin/GstNvInfer:primary_gie:
Config file path: /opt/nvidia/deepstream/deepstream-6.3/sources/deepstream_python_apps/apps/deepstream-test3/calib/DeepStream-Yolo/config_infer_primary_yoloV8.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED
App run failed

obviously, the calib.table file was not found (because it is not kek)

Therefore, I'm going to INT8Calibration.md

I'm doing apt-get install libopencv-dev

then I do clean just in case CUDA_VER=12.1 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo clean

then CUDA_VER=12.1 OPENCV=1 make -C nvdsinfer_custom_impl_Yolo

I get the following output

make: Entering directory '/opt/nvidia/deepstream/deepstream-6.3/sources/deepstream_python_apps/apps/deepstream-test3/calib/DeepStream-Yolo/nvdsinfer_custom_impl_Yolo'
g++ -c -DOPENCV -o utils.o -Wall -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include -I/usr/include/opencv4/opencv -I/usr/include/opencv4 utils.cpp
g++ -c -DOPENCV -o yolo.o -Wall -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include -I/usr/include/opencv4/opencv -I/usr/include/opencv4 yolo.cpp
yolo.cpp: In member function 'nvinfer1::ICudaEngine* Yolo::createEngine(nvinfer1::IBuilder*, nvinfer1::IBuilderConfig*)':
yolo.cpp:207:83: warning: 'nvinfer1::ICudaEngine* nvinfer1::IBuilder::buildEngineWithConfig(nvinfer1::INetworkDefinition&, nvinfer1::IBuilderConfig&)' is deprecated [-Wdeprecated-declarations]
  207 |   nvinfer1::ICudaEngine* engine = builder->buildEngineWithConfig(*network, *config);
      |                                                                                   ^
In file included from /usr/include/x86_64-linux-gnu/NvOnnxParser.h:26,
                 from yolo.cpp:26:
/usr/include/x86_64-linux-gnu/NvInfer.h:9483:43: note: declared here
 9483 |     TRT_DEPRECATED nvinfer1::ICudaEngine* buildEngineWithConfig(
      |                                           ^~~~~~~~~~~~~~~~~~~~~
g++ -c -DOPENCV -o nvdsinfer_yolo_engine.o -Wall -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include -I/usr/include/opencv4/opencv -I/usr/include/opencv4 nvdsinfer_yolo_engine.cpp
g++ -c -DOPENCV -o nvdsinitinputlayers_Yolo.o -Wall -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include -I/usr/include/opencv4/opencv -I/usr/include/opencv4 nvdsinitinputlayers_Yolo.cpp
g++ -c -DOPENCV -o nvdsparsebbox_Yolo.o -Wall -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include -I/usr/include/opencv4/opencv -I/usr/include/opencv4 nvdsparsebbox_Yolo.cpp
g++ -c -DOPENCV -o yoloPlugins.o -Wall -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include -I/usr/include/opencv4/opencv -I/usr/include/opencv4 yoloPlugins.cpp
g++ -c -DOPENCV -o calibrator.o -Wall -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include -I/usr/include/opencv4/opencv -I/usr/include/opencv4 calibrator.cpp
g++ -c -DOPENCV -o layers/shortcut_layer.o -Wall -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include -I/usr/include/opencv4/opencv -I/usr/include/opencv4 layers/shortcut_layer.cpp
g++ -c -DOPENCV -o layers/batchnorm_layer.o -Wall -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include -I/usr/include/opencv4/opencv -I/usr/include/opencv4 layers/batchnorm_layer.cpp
g++ -c -DOPENCV -o layers/convolutional_layer.o -Wall -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include -I/usr/include/opencv4/opencv -I/usr/include/opencv4 layers/convolutional_layer.cpp
g++ -c -DOPENCV -o layers/slice_layer.o -Wall -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include -I/usr/include/opencv4/opencv -I/usr/include/opencv4 layers/slice_layer.cpp
g++ -c -DOPENCV -o layers/pooling_layer.o -Wall -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include -I/usr/include/opencv4/opencv -I/usr/include/opencv4 layers/pooling_layer.cpp
g++ -c -DOPENCV -o layers/reorg_layer.o -Wall -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include -I/usr/include/opencv4/opencv -I/usr/include/opencv4 layers/reorg_layer.cpp
g++ -c -DOPENCV -o layers/activation_layer.o -Wall -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include -I/usr/include/opencv4/opencv -I/usr/include/opencv4 layers/activation_layer.cpp
g++ -c -DOPENCV -o layers/channels_layer.o -Wall -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include -I/usr/include/opencv4/opencv -I/usr/include/opencv4 layers/channels_layer.cpp
g++ -c -DOPENCV -o layers/sam_layer.o -Wall -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include -I/usr/include/opencv4/opencv -I/usr/include/opencv4 layers/sam_layer.cpp
g++ -c -DOPENCV -o layers/deconvolutional_layer.o -Wall -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include -I/usr/include/opencv4/opencv -I/usr/include/opencv4 layers/deconvolutional_layer.cpp
g++ -c -DOPENCV -o layers/route_layer.o -Wall -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include -I/usr/include/opencv4/opencv -I/usr/include/opencv4 layers/route_layer.cpp
g++ -c -DOPENCV -o layers/upsample_layer.o -Wall -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include -I/usr/include/opencv4/opencv -I/usr/include/opencv4 layers/upsample_layer.cpp
g++ -c -DOPENCV -o layers/implicit_layer.o -Wall -std=c++11 -shared -fPIC -Wno-error=deprecated-declarations -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include -I/usr/include/opencv4/opencv -I/usr/include/opencv4 layers/implicit_layer.cpp
/usr/local/cuda-12.1/bin/nvcc -c -o yoloForward_v2.o --compiler-options '-fPIC' -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include yoloForward_v2.cu
/usr/local/cuda-12.1/bin/nvcc -c -o yoloForward.o --compiler-options '-fPIC' -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include yoloForward.cu
/usr/local/cuda-12.1/bin/nvcc -c -o yoloForward_nc.o --compiler-options '-fPIC' -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include yoloForward_nc.cu
/usr/local/cuda-12.1/bin/nvcc -c -o nvdsparsebbox_Yolo_cuda.o --compiler-options '-fPIC' -I/opt/nvidia/deepstream/deepstream/sources/includes -I/usr/local/cuda-12.1/include nvdsparsebbox_Yolo_cuda.cu
g++ -o libnvdsinfer_custom_impl_Yolo.so  utils.o yolo.o nvdsinfer_yolo_engine.o nvdsinitinputlayers_Yolo.o nvdsparsebbox_Yolo.o yoloPlugins.o calibrator.o layers/shortcut_layer.o layers/batchnorm_layer.o layers/convolutional_layer.o layers/slice_layer.o layers/pooling_layer.o layers/reorg_layer.o layers/activation_layer.o layers/channels_layer.o layers/sam_layer.o layers/deconvolutional_layer.o layers/route_layer.o layers/upsample_layer.o layers/implicit_layer.o yoloForward_v2.o yoloForward.o yoloForward_nc.o nvdsparsebbox_Yolo_cuda.o -shared -Wl,--start-group -lopencv_stitching -lopencv_aruco -lopencv_bgsegm -lopencv_bioinspired -lopencv_ccalib -lopencv_dnn_objdetect -lopencv_dnn_superres -lopencv_dpm -lopencv_highgui -lopencv_face -lopencv_freetype -lopencv_fuzzy -lopencv_hdf -lopencv_hfs -lopencv_img_hash -lopencv_line_descriptor -lopencv_quality -lopencv_reg -lopencv_rgbd -lopencv_saliency -lopencv_shape -lopencv_stereo -lopencv_structured_light -lopencv_phase_unwrapping -lopencv_superres -lopencv_optflow -lopencv_surface_matching -lopencv_tracking -lopencv_datasets -lopencv_text -lopencv_dnn -lopencv_plot -lopencv_ml -lopencv_videostab -lopencv_videoio -lopencv_viz -lopencv_ximgproc -lopencv_video -lopencv_xobjdetect -lopencv_objdetect -lopencv_calib3d -lopencv_imgcodecs -lopencv_features2d -lopencv_flann -lopencv_xphoto -lopencv_photo -lopencv_imgproc -lopencv_core -lnvinfer_plugin -lnvinfer -lnvparsers -lnvonnxparser -L/usr/local/cuda-12.1/lib64 -lcudart -lcublas -lstdc++fs -Wl,--end-group
make: Leaving directory '/opt/nvidia/deepstream/deepstream-6.3/sources/deepstream_python_apps/apps/deepstream-test3/calib/DeepStream-Yolo/nvdsinfer_custom_impl_Yolo'

at this stage, I still don't have calib.table and model_b1_gpu0_int8.engine next, I create a folder with calibration images and create a txt file with paths to these images setting the export variables

export INT8_CALIB_IMG_PATH=calibration.txt 
export INT8_CALIB_BATCH_SIZE=1

Next it says Edit the config_infer file

I have not found any file called config_infer, there is a file called config_infer_primary.txt that's why I'm editing it.

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
model-color-format=0
# custom-network-config=yolov4.cfg
# model-file=yolov4.weights
onnx-file=/opt/nvidia/deepstream/deepstream-6.3/sources/deepstream_python_apps/apps/deepstream-test3/calib/DeepStream-Yolo/testbest.onnx
model-engine-file=model_b1_gpu0_int8.engine
int8-calib-file=calib.table
labelfile-path=labels.txt
batch-size=1
network-mode=1
num-detected-classes=3
interval=0
gie-unique-id=1
process-mode=1
network-type=0
cluster-mode=2
maintain-aspect-ratio=0
symmetric-padding=1
#force-implicit-batch-dim=1
#workspace-size=2000
parse-bbox-func-name=NvDsInferParseYolo
#parse-bbox-func-name=NvDsInferParseYoloCuda
custom-lib-path=nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so
engine-create-func-name=NvDsInferYoloCudaEngineGet

[class-attrs-all]
nms-iou-threshold=0.45
pre-cluster-threshold=0.25
topk=300

after that, it starts calibration and outputs the following

WARNING: [TRT]: CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars
WARNING: ../nvdsinfer/nvdsinfer_model_builder.cpp:659 INT8 calibration file not specified/accessible. INT8 calibration can be done through setDynamicRange API in 'NvDsInferCreateNetwork' implementation
WARNING: [TRT]: onnx2trt_utils.cpp:377: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
WARNING: [TRT]: onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
WARNING: [TRT]: Tensor DataType is determined at build time for tensors not marked as input or output.

Building the TensorRT Engine

File does not exist: /opt/nvidia/deepstream/deepstream-6.3/sources/deepstream_python_apps/apps/deepstream-test3/calib/DeepStream-Yolo/calib.table
WARNING: [TRT]: CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars
Load image: /opt/nvidia/deepstream/deepstream-6.3/sources/deepstream_python_apps/apps/deepstream-test3/calib/DeepStream-Yolo/calibration/frame000042_07_18_2023_03_00_457947_SRC.jpg
Progress: 0.1%
Load image: /opt/nvidia/deepstream/deepstream-6.3/sources/deepstream_python_apps/apps/deepstream-test3/calib/DeepStream-Yolo/calibration/frame000042_07_18_2023_03_00_571936_SRC.jpg
Progress: 0.2%
Load image: /opt/nvidia/deepstream/deepstream-6.3/sources/deepstream_python_apps/apps/deepstream-test3/calib/DeepStream-Yolo/calibration/frame000042_07_18_2023_03_00_780444_SRC.jpg

Please pay attention to the warnings, they will be throughout the work

After that, I still get calib.table and model_b1_gpu0_int8.engine I run my deepstream pipline with the created calibration files, my config looks like this:

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
model-color-format=0
onnx-file=/opt/nvidia/deepstream/deepstream-6.3/sources/deepstream_python_apps/apps/deepstream-test3/calib/DeepStream-Yolo/testbest.onnx
model-engine-file=model_b1_gpu0_int8.engine
int8-calib-file=calib.table
labelfile-path=labels.txt
batch-size=1
network-mode=1
num-detected-classes=3
interval=0
gie-unique-id=1
process-mode=1
network-type=0
cluster-mode=2
maintain-aspect-ratio=1
symmetric-padding=1
#workspace-size=2000
parse-bbox-func-name=NvDsInferParseYolo
#parse-bbox-func-name=NvDsInferParseYoloCuda
custom-lib-path=/opt/nvidia/deepstream/deepstream-6.3/sources/deepstream_python_apps/apps/deepstream-test3/calib/DeepStream-Yolo/nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so
engine-create-func-name=NvDsInferYoloCudaEngineGet

[class-attrs-all]
nms-iou-threshold=0.45
pre-cluster-threshold=0.25
topk=300

WARNING: [TRT]: CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars
WARNING: [TRT]: onnx2trt_utils.cpp:377: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
WARNING: [TRT]: onnx2trt_utils.cpp:403: One or more weights outside the range of INT32 was clamped
WARNING: [TRT]: Tensor DataType is determined at build time for tensors not marked as input or output.

Building the TensorRT Engine

WARNING: [TRT]: Missing scale and zero-point for tensor /0/model.22/Squeeze_output_0, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor /0/model.22/Squeeze_1_output_0, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor /0/model.22/Squeeze_2_output_0, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 367) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 368) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 378) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 382) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 383) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 393) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor /0/model.22/Reshape_output_0, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor /0/model.22/Reshape_1_output_0, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 442) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 443) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 453) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 457) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 458) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 468) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor /0/model.22/Reshape_5_output_0, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor /0/model.22/Reshape_6_output_0, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 517) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 518) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 528) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 532) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 533) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 543) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor /0/model.22/Reshape_10_output_0, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor /0/model.22/Reshape_11_output_0, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor /0/model.22/dfl/Softmax_output_0, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor (Unnamed Layer* 705) [Constant]_output, expect fall back to non-int8 implementation for any layer consuming or producing given tensor
WARNING: [TRT]: Missing scale and zero-point for tensor /1/ArgMax_output_0, expect fall back to non-int8 implementation for any layer consuming or producing given tensor

and I get very serious warnings, also the pipeline runs for a very long time, and when it does run, the performance is very low on the order of 2-3 FPS Moreover, it works faster without calibration

How to fix it, what is the problem?

IronmanVsThanos commented 5 months ago

https://github.com/marcoslucianops/DeepStream-Yolo/issues/504 I guess i meet same error like U ，but i do not known How to fix it，can u help me ？thank U!