NVIDIA-AI-IOT / deepstream_tao_apps

Sample apps to demonstrate how to deploy models trained with TAO on DeepStream
MIT License
369 stars 95 forks source link

deepstream-heartrate-app: Assertion `bindingDims.numElements > 0' failed. #88

Open zenssh opened 8 months ago

zenssh commented 8 months ago

Branch: main System: Ubuntu 22.04 DeepStream: 6.4.0 TensorRT: 8.6.1 CUDA: 12.2 Nvidia Driver: 535.104.05

I'm getting an Assertion 'bindingDims.numElements > 0' failed error when running deepstream-heartrate-app.


$ ./deepstream-heartrate-app 1 file:///data2/HeartRateNet/deepstream_tao_apps/apps/tao_others/deepstream-heartrate-app/ppg72.avi ./heartrate
Now playing: file:///data2/HeartRateNet/deepstream_tao_apps/apps/tao_others/deepstream-heartrate-app/ppg72.avi
Inside Custom Lib : Setting Prop Key=config-file Value=../../../../configs/nvinfer/heartrate_tao/sample_heartrate_model_config.txt
0:00:02.143789140 96206 0x55a2b21a9470 WARN                 nvinfer gstnvinfer.cpp:679:gst_nvinfer_logger:<primary-infer-engine1> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1243> [UID = 1]: Warning, OpenCV has been deprecated. Using NMS for clustering instead of cv::groupRectangles with topK = 20 and NMS Threshold = 0.5
0:00:09.009537279 96206 0x55a2b21a9470 INFO                 nvinfer gstnvinfer.cpp:682:gst_nvinfer_logger:<primary-infer-engine1> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:2092> [UID = 1]: deserialized trt engine from :/data2/HeartRateNet/deepstream_tao_apps/models/facenet/facenet.etlt_b1_gpu0_int8.engine
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:610 [Implicit Engine Info]: layers num: 3
0   INPUT  kFLOAT input_1         3x416x736
1   OUTPUT kFLOAT output_bbox/BiasAdd 4x26x46
2   OUTPUT kFLOAT output_cov/Sigmoid 1x26x46

0:00:09.134070505 96206 0x55a2b21a9470 INFO                 nvinfer gstnvinfer.cpp:682:gst_nvinfer_logger:<primary-infer-engine1> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2195> [UID = 1]: Use deserialized engine model: /data2/HeartRateNet/deepstream_tao_apps/models/facenet/facenet.etlt_b1_gpu0_int8.engine
0:00:09.142766853 96206 0x55a2b21a9470 INFO                 nvinfer gstnvinfer_impl.cpp:328:notifyLoadModelStatus:<primary-infer-engine1> [UID 1]: Load new model:../../../configs/nvinfer/facial_tao/config_infer_primary_facenet.txt sucessfully
Decodebin child added: source
Decodebin child added: decodebin0
Running...
Decodebin child added: avidemux0
Decodebin child added: multiqueue0
Decodebin child added: mpeg4vparse0
Decodebin child added: nvv4l2decoder0
In cb_newpad
###Decodebin pick nvidia decoder plugin.
HeartRate model config file: heartrate.engine
WARNING: Max GPU memory (0.000000) is less than or equal to 0, falling back to 99
WARNING: [TRT]: onnx2trt_utils.cpp:374: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
WARNING: [TRT]: TensorRT encountered issues when converting weights between types and that could affect accuracy.
WARNING: [TRT]: If this is not the desired behavior, please modify the weights or retrain with regularization to adjust the magnitude of the weights.
WARNING: [TRT]: Check verbose logs for the list of affected weights.
WARNING: [TRT]: - 10 weights are affected by this issue: Detected subnormal FP16 values.
WARNING: ../nvdsinfer/nvdsinfer_model_builder.cpp:1459 Serialize engine failed because of file path: ./heartrateinfer_impl/../../../../models/heartrate/heartrate.etlt_b16_gpu0_fp16.engine opened error
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:610 [FullDims Engine Info]: layers num: 3
0   INPUT  kFLOAT motion_input:0  3x72x72         min: 1x3x72x72       opt: 16x3x72x72      Max: 16x3x72x72
1   INPUT  kFLOAT appearance_input:0 3x72x72         min: 1x3x72x72       opt: 16x3x72x72      Max: 16x3x72x72
2   OUTPUT kFLOAT lambda_1/Squeeze:0 0               min: 0               opt: 0               Max: 0

deepstream-heartrate-app: nvdsinfer_context_impl.cpp:1446: NvDsInferStatus nvdsinfer::NvDsInferContextImpl::resizeOutputBufferpool(uint32_t): Assertion `bindingDims.numElements > 0' failed.
Aborted (core dumped)