自己训练的模型，c++ 运行picodet报错。

LightSun commented 1 year ago

环境

【FastDeploy版本】： 1.0.2

【编译命令】如果您是自行编译的FastDeploy，请说明您的编译方式（参数命令）

git clone https://github.com/PaddlePaddle/FastDeploy.git
cd FastDeploy
mkdir build 
cd build
cmake .. -DENABLE_ORT_BACKEND=OFF \
    -DENABLE_PADDLE_BACKEND=OFF \
    -DENABLE_OPENVINO_BACKEND=ON \
    -DCMAKE_INSTALL_PREFIX=${PWD}/compiled_fastdeploy_sdk \
    -DENABLE_VISION=ON \
    -DOPENCV_DIRECTORY=${OPENCV4} \
    -DENABLE_TEXT=ON
make -j8
make install

【系统平台】: Linux x64(Ubuntu 18.04)
【硬件】：说明具体硬件型号. 使用的openvino 作为后端
【编译语言】： C++

log

[INFO] fastdeploy/vision/common/processors/transform.cc(45)::FuseNormalizeCast  Normalize and Cast are fused to Normalize in preprocessing pipeline.
[INFO] fastdeploy/vision/common/processors/transform.cc(93)::FuseNormalizeHWC2CHW   Normalize and HWC2CHW are fused to NormalizeAndPermute  in preprocessing pipeline.
[INFO] fastdeploy/vision/common/processors/transform.cc(159)::FuseNormalizeColorConvert BGR2RGB and NormalizeAndPermute are fused to NormalizeAndPermute with swap_rb=1
terminate called after throwing an instance of 'ov::Exception'
what():  [ NETWORK_NOT_READ ] Unable to read the model: /xxx/heaven7/libs/paddle/paddle_ocr_module/icu_cls/model.pdmodel Please check that model format: pdmodel is supported and the model is correct. Available frontends: 
13:24:48: The program has unexpectedly finished.

参考代码, picodet

jiangjiajun commented 1 year ago

看起来是提示加载的模型文件不正确，如果确认模型文件没问题，你可以试下使用其它的后端（当前是OpenVINO后端，可以改为Paddle后端）

LightSun commented 1 year ago

@jiangjiajun i found a cmake log:

 -- Could NOT find TBB (missing: TBB_DIR)

may be this reason?

jiangjiajun commented 1 year ago

这个日志没有影响

LightSun commented 1 year ago

我测试了下。paddle作为后端是正常的，openvino有问题。难道是某些算子openvino不支持？

jiangjiajun commented 1 year ago

你的Paddle版本是多少，尝试使用Paddle 2.3版本重新导出模型试下，同时也可以附上你的模型我来看下

LightSun commented 1 year ago

-- *************FastDeploy Building Summary**********
--   CMake version             : 3.18.0
--   CMake command             : /usr/local/bin/cmake
--   System                    : Linux
--   C++ compiler              : /usr/bin/g++
--   C++ compiler version      : 8.4.0
--   CXX flags                 : -Wno-format
--   WITH_GPU                  : OFF
--   ENABLE_ORT_BACKEND        : ON
--   ENABLE_RKNPU2_BACKEND     : OFF
--   ENABLE_PADDLE_BACKEND     : ON
--   ENABLE_POROS_BACKEND      : OFF
--   ENABLE_OPENVINO_BACKEND   : ON
--   ENABLE_TRT_BACKEND        : OFF
--   ENABLE_LITE_BACKEND       : OFF
--   Paddle Inference version  : 2.4-dev3
--   OpenVINO version          : 2022.2.0.dev20220829
--   ENABLE_TRT_BACKEND        : OFF
--   ENABLE_VISION             : ON
--   ENABLE_TEXT               : ON
--   OPENCV_DIRECTORY          : /home/heaven7/heaven7/libs/build-opencv-3.4.5-Desktop_Qt_5_14_2_GCC_64bit-Release
--   DEPENDENCY_LIBS           : /home/heaven7/heaven7/libs/paddle/FastDeploy/compiled_fastdeploy_sdk/lib/libfastdeploy.so;/home/heaven7/heaven7/libs/paddle/FastDeploy/compiled_fastdeploy_sdk/third_libs/install/onnxruntime/lib/libonnxruntime.so;/home/heaven7/heaven7/libs/paddle/FastDeploy/compiled_fastdeploy_sdk/third_libs/install/paddle_inference/paddle/lib/libpaddle_inference.so;/home/heaven7/heaven7/libs/paddle/FastDeploy/compiled_fastdeploy_sdk/third_libs/install/paddle_inference/third_party/install/mkldnn/lib/libmkldnn.so.0;/home/heaven7/heaven7/libs/paddle/FastDeploy/compiled_fastdeploy_sdk/third_libs/install/paddle_inference/third_party/install/mklml/lib/libiomp5.so;/home/heaven7/heaven7/libs/paddle/FastDeploy/compiled_fastdeploy_sdk/third_libs/install/openvino/runtime/lib/libopenvino.so;opencv_calib3d;opencv_core;opencv_dnn;opencv_features2d;opencv_flann;opencv_highgui;opencv_imgcodecs;opencv_imgproc;opencv_ml;opencv_objdetect;opencv_photo;opencv_shape;opencv_stitching;opencv_superres;opencv_video;opencv_videoio;opencv_videostab;/home/heaven7/heaven7/libs/paddle/FastDeploy/compiled_fastdeploy_sdk/third_libs/install/fast_tokenizer/lib/libcore_tokenizers.so;/home/heaven7/heaven7/libs/paddle/FastDeploy/compiled_fastdeploy_sdk/third_libs/install/paddle2onnx/lib/libpaddle2onnx.so

LightSun commented 1 year ago

@jiangjiajun 我发现好像找到问题了。我通过sh 导出LD_LIBRARY_PATH是可以跑openvino版本的，但是复制so到可执行文件目录就出问题了。可能so和系统内的其它有冲突

LIB_DIR=/home/heaven7/heaven7/libs/paddle/FastDeploy/FastDeploy/build/compiled_fastdeploy_sdk
LD1=${LIB_DIR}/third_libs/install/fast_tokenizer/lib
LD2=${LIB_DIR}/third_libs/install/onnxruntime/lib
LD3=${LIB_DIR}/third_libs/install/openvino/runtime/3rdparty/omp/lib
LD4=${LIB_DIR}/third_libs/install/openvino/runtime/lib
LD5=${LIB_DIR}/third_libs/install/paddle2onnx/lib
LD6=${LIB_DIR}/third_libs/install/paddle_inference/paddle/lib
export LD_LIBRARY_PATH=${LD1}:${LD2}:${LD3}:${LD4}:${LD5}:${LD6}
./test_medicu

LightSun commented 1 year ago

sh test_medicu.sh 
[INFO] fastdeploy/vision/common/processors/transform.cc(45)::FuseNormalizeCast  Normalize and Cast are fused to Normalize in preprocessing pipeline.
[INFO] fastdeploy/vision/common/processors/transform.cc(93)::FuseNormalizeHWC2CHW   Normalize and HWC2CHW are fused to NormalizeAndPermute  in preprocessing pipeline.
[INFO] fastdeploy/vision/common/processors/transform.cc(159)::FuseNormalizeColorConvert BGR2RGB and NormalizeAndPermute are fused to NormalizeAndPermute with swap_rb=1
[INFO] fastdeploy/backends/openvino/ov_backend.cc(199)::InitFromPaddle  Compile OpenVINO model on device_name:CPU.
[INFO] fastdeploy/runtime.cc(611)::Init Runtime initialized with Backend::OPENVINO in Device::CPU.
[INFO] fastdeploy/backends/openvino/ov_backend.cc(199)::InitFromPaddle  Compile OpenVINO model on device_name:CPU.
[INFO] fastdeploy/runtime.cc(611)::Init Runtime initialized with Backend::OPENVINO in Device::CPU.
[INFO] fastdeploy/backends/openvino/ov_backend.cc(199)::InitFromPaddle  Compile OpenVINO model on device_name:CPU.
[INFO] fastdeploy/runtime.cc(611)::Init Runtime initialized with Backend::OPENVINO in Device::CPU.
DetectionResult: [xmin, ymin, xmax, ymax, score, label_id]
1435.658447,66.285675, 1694.272949, 218.564667, 0.990556, 0
1263.595215,909.950195, 1346.617920, 977.714600, 0.042262, 0
1765.409546,230.855072, 1915.470459, 294.940308, 0.040389, 0
838.823242,699.672424, 1141.680420, 863.212158, 0.034771, 0
1441.908081,679.837158, 1679.225586, 840.638184, 0.031623, 0
1415.420898,-8.434410, 1665.532104, 70.056984, 0.031083, 0
1394.026733,-12.360642, 1575.871948, 71.894745, 0.029087, 0
478.777100,815.130554, 1246.014893, 979.663818, 0.028912, 0
477.554382,814.834351, 1246.204224, 979.641846, 0.987581, 1
-268.798553,-2.760109, 2176.470947, 1102.004517, 0.029999, 1
1442.597412,679.767090, 1679.497192, 841.239685, 0.027024, 1
1760.458740,305.405304, 1891.878296, 367.898163, 0.026352, 1
-457.519043,191.845398, 2377.245361, 1363.046875, 0.025707, 1
1444.363770,521.728577, 1680.372559, 679.296631, 0.987899, 2
1765.409546,230.855072, 1915.470459, 294.940308, 0.045830, 2
1442.597412,679.767090, 1679.497192, 841.239685, 0.035877, 2
1263.365601,910.199646, 1346.472900, 978.024902, 0.035680, 2
476.936279,820.978149, 1061.842285, 980.125305, 0.031167, 2
1760.783936,305.530762, 1891.575684, 367.983063, 0.028310, 2
58.392792,-192.717697, 1422.047729, 711.935059, 0.027780, 2
-141.600677,-5.904705, 2628.126709, 1287.326416, 0.027415, 2
128.397034,-60.456047, 1819.821899, 895.833862, 0.025684, 2
838.823242,699.672424, 1141.680420, 863.212158, 0.025595, 2
-643.981079,-112.013885, 1637.936279, 1139.852295, 0.025100, 2
1437.524170,221.504120, 1729.766357, 364.717102, 0.984222, 3
1764.973389,230.962738, 1915.859131, 295.036072, 0.036077, 3
1263.365601,910.199646, 1346.472900, 978.024902, 0.034074, 3
1760.458740,305.405304, 1891.878296, 367.898163, 0.972184, 4
1263.746094,909.956238, 1346.572998, 977.522583, 0.964761, 4
-9.728485,-47.704319, 1045.250000, 812.092834, 0.036621, 4
478.777100,815.130554, 1246.014893, 979.663818, 0.029566, 4
1765.006104,230.742035, 1915.377319, 294.920319, 0.028536, 4
-268.798553,-2.760109, 2176.470947, 1102.004517, 0.028233, 4
1232.375244,824.160156, 1372.042603, 896.160645, 0.026874, 4
1436.847290,373.824707, 1841.336060, 522.176453, 0.061885, 5
478.777100,815.130554, 1246.014893, 979.663818, 0.036285, 5
-268.798553,-2.760109, 2176.470947, 1102.004517, 0.028784, 5
-457.519043,191.845398, 2377.245361, 1363.046875, 0.027073, 5
1437.931274,221.854980, 1729.566650, 364.642792, 0.026008, 5
478.777100,815.130554, 1246.014893, 979.663818, 0.034648, 6
-141.600677,-5.904705, 2628.126709, 1287.326416, 0.026190, 6
1442.632568,680.193542, 1679.068359, 841.036438, 0.985030, 7
1263.365601,910.199646, 1346.472900, 978.024902, 0.037699, 7
1764.973389,230.962738, 1915.859131, 295.036072, 0.034263, 7
477.745667,815.188110, 1246.028809, 979.749268, 0.030776, 7
-268.798553,-2.760109, 2176.470947, 1102.004517, 0.029566, 7
-457.519043,191.845398, 2377.245361, 1363.046875, 0.025871, 7
1765.006104,230.742035, 1915.377319, 294.920319, 0.981108, 8
1263.595215,909.950195, 1346.617920, 977.714600, 0.038422, 8
-268.798553,-2.760109, 2176.470947, 1102.004517, 0.029677, 8
478.777100,815.130554, 1246.014893, 979.663818, 0.028849, 8
1441.908081,679.837158, 1679.225586, 840.638184, 0.027188, 8
237.346298,1015.060608, 401.469055, 1075.008789, 0.027083, 8
328.970581,1031.309326, 530.176636, 1077.674072, 0.026468, 8
-457.519043,191.845398, 2377.245361, 1363.046875, 0.025787, 8
1436.570068,373.838684, 1841.844482, 521.972595, 0.995088, 9
1428.173218,249.757034, 2102.090820, 406.833099, 0.052694, 9
-268.798553,-2.760109, 2176.470947, 1102.004517, 0.029692, 9
-0.559296,-49.066711, 1254.861572, 821.370239, 0.027751, 9
1441.908081,679.837158, 1679.225586, 840.638184, 0.026671, 9
-457.519043,191.845398, 2377.245361, 1363.046875, 0.025635, 9

Visualized result saved in ./vis_result.jpg
infer cost total:  1218.161
cls = 0, text = ECG
cls = 1, text = @15:38 NIBP
cls = 2, text = Resp
cls = 3, text = Sp02
cls = 4, text = 86
cls = 4, text = 85
cls = 7, text = CVP
cls = 8, text = 2.69
cls = 9, text = BAI

PaddlePaddle / FastDeploy

自己训练的模型，c++ 运行picodet报错。 #1046

环境