search

PaddlePaddle / FastDeploy

⚡️An Easy-to-use and Fast Deep Learning Model Deployment Toolkit for ☁️Cloud 📱Mobile and 📹Edge. Including Image, Video, Text and Audio 20+ main stream scenarios and 150+ SOTA models with end-to-end optimization, multi-platform and multi-framework support.
https://www.paddlepaddle.org.cn/fastdeploy
Apache License 2.0
2.92k stars 455 forks source link

rk3568,rk3588 跑自己转换的rknn例子模型PP_TinyPose, picodet 中的例子 Invalid RKNN model version 6 #2211

Closed erroot closed 11 months ago

erroot commented 12 months ago

温馨提示：根据社区不完全统计，按照模板提问，可以加快回复和解决问题的速度

环境

问题日志及出现问题的操作流程

cpu测试中间onnx 模型文件是正常加载推理结果正常，因此推测是“ONNX模型转RKNN模型”出错。 python tools/rknpu2/export.py --config_path tools/rknpu2/config/picodet_s_416_coco_lcnet_unquantized.yaml \ --target_platform rk3588 但是转换调试输出没有报错 D RKNN: [12:36:04.936] --------------------------------------------------------------------------------------+--------------------------------- D RKNN: [12:36:04.938] ---------------------------------------- D RKNN: [12:36:04.938] Total Weight Memory Size: 2641920 D RKNN: [12:36:04.938] Total Internal Memory Size: 6230016 D RKNN: [12:36:04.938] Predict Internal Memory RW Amount: 175689408 D RKNN: [12:36:04.938] Predict Weight Memory RW Amount: 2755696 D RKNN: [12:36:04.938] ---------------------------------------- D RKNN: [12:36:04.938] <<<<<<<< end: N4rknn21RKNNMemStatisticsPassE I rknn buiding done. W init_runtime: Target is None, use simulator! Export OK!

图片

RK3568 ： root@RK356X:/userdata/app_data/pp/TinyPos# ./infer_tinypose_demo PP_TinyPose_256x192_infer hrnet_demo.jpg PP_TinyPose_256x192_infer/PP_TinyPose_256x192_infer_rk3588_unquantized.rknn E RKNN: [15:17:54.643] 6, 1 E RKNN: [15:17:54.643] Invalid RKNN model version 6 E RKNN: [15:17:54.643] rknn_init, load model failed! [ERROR] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(180)::LoadModel The function(rknn_init) failed! ret=-6 [ERROR] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(123)::Init Load model failed [ERROR] fastdeploy/runtime/runtime.cc(361)::CreateRKNPU2Backend Failed to initialize RKNPU2 backend. Aborted

RK3588: root@firefly:/opt/pp/FastDeploy-develop/examples/vision/keypointdetection/tiny_pose/rknpu2/cpp/build# ./infer_tinypose_demo ./PP_TinyPose_256x192_infer ./hrnet_demo.jpg ./PP_TinyPose_256x192_infer/PP_TinyPose_256x192_infer_rk3588_unquantized.rknn E RKNN: [10:30:34.599] 6, 1 E RKNN: [10:30:34.600] Invalid RKNN model version 6 E RKNN: [10:30:34.600] rknn_init, load model failed! [ERROR] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(180)::LoadModel The function(rknn_init) failed! ret=-6 [ERROR] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(123)::Init Load model failed [ERROR] fastdeploy/runtime/runtime.cc(361)::CreateRKNPU2Backend Failed to initialize RKNPU2 backend.

erroot commented 12 months ago

模型转换的结果没有报错

D RKNN: [07:04:41.390] -------------------------------------------------------------------------------------------+--------------------------------- D RKNN: [07:04:41.665] ---------------------------------------- D RKNN: [07:04:41.665] Total Weight Memory Size: 4493952 D RKNN: [07:04:41.665] Total Internal Memory Size: 1867776 D RKNN: [07:04:41.665] Predict Internal Memory RW Amount: 151041034 D RKNN: [07:04:41.665] Predict Weight Memory RW Amount: 4486528 D RKNN: [07:04:41.665] ---------------------------------------- D RKNN: [07:04:41.665] <<<<<<<< end: N4rknn21RKNNMemStatisticsPassE I rknn buiding done. W init_runtime: Target is None, use simulator! Export OK!

erroot commented 12 months ago

找到原因了，看文档不仔细啊，实际安装版本是rknn_toolkit2-1.5.1b19，但是FastDeploy 默认librknnrt.so 库是1.4.2b3 这个版本统一起来就可以正常在3568上rknn 正常推理了，根据文档到网盘中下载和转换工具相同版本的librknnrt.so 图片

结果 root@RK356X:/userdata/app_data/pp/ppdete# ./infer_picodet_demo picodet_s_416_coco_lcnet 000000014439.jpg 1 [INFO] fastdeploy/vision/common/processors/transform.cc(45)::FuseNormalizeCast Normalize and Cast are fused to Normalize in preprocessing pipeline. [INFO] fastdeploy/vision/common/processors/transform.cc(93)::FuseNormalizeHWC2CHW Normalize and HWC2CHW are fused to NormalizeAndPermute in preprocessing pipeline. [INFO] fastdeploy/vision/common/processors/transform.cc(159)::FuseNormalizeColorConvert BGR2RGB and NormalizeAndPermute are fused to NormalizeAndPermute with swap_rb=1 [INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(81)::GetSDKAndDeviceVersion rknpu2 runtime version: 1.5.1b19 (32afb0e92@2023-07-14T12:46:17) [INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(82)::GetSDKAndDeviceVersion rknpu2 driver version: 0.4.2 index=0, name=image, n_dims=4, dims=[1, 416, 416, 3], n_elems=519168, size=1038336, fmt=NHWC, type=FP16, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0 index=0, name=p2o.Mul.179, n_dims=4, dims=[1, 3598, 4, 1], n_elems=14392, size=28784, fmt=NCHW, type=FP32, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0 index=1, name=p2o.Concat.9, n_dims=4, dims=[1, 80, 3598, 1], n_elems=287840, size=575680, fmt=NCHW, type=FP32, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0 [INFO] fastdeploy/runtime/runtime.cc(367)::CreateRKNPU2Backend Runtime initialized with Backend::RKNPU2 in Device::RKNPU. [INFO] fastdeploy/vision/common/processors/transform.cc(159)::FuseNormalizeColorConvert BGR2RGB and Normalize are fused to Normalize with swap_rb=1 [WARNING] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(420)::InitRKNNTensorMemory The input tensor type != model's inputs type.The input_type need FP16,but inputs[0].type is UINT8 [FastDeploy] PPDet in RKNPU2 duration = 0.131852s. DetectionResult: [xmin, ymin, xmax, ymax, score, label_id] 413.461548,89.649635, 508.461548, 282.605743, 0.833008, 0 160.961548,81.698311, 200.000000, 166.795670, 0.812012, 0 265.576935,79.756004, 299.230774, 167.524033, 0.791992, 0 105.384621,46.251198, 126.730774, 93.534248, 0.769531, 0 584.230774,113.685692, 612.692322, 178.085327, 0.762695, 0 328.269257,40.211838, 344.423096, 80.545067, 0.636719, 0 379.038483,42.093449, 396.153870, 83.397835, 0.554199, 0 510.000031,116.052879, 598.846191, 278.235565, 0.541504, 0 24.038462,116.417061, 55.048080, 153.442307, 0.443115, 0 58.557693,136.325714, 107.115387, 173.836533, 0.428955, 0 352.307709,45.097954, 376.923096, 104.034851, 0.421387, 0 188.750000,45.795971, 200.000000, 61.637917, 0.414551, 0 352.692322,45.097954, 369.615387, 87.343147, 0.388672, 0 360.961548,61.000599, 383.461548, 114.353363, 0.363525, 0 505.000031,114.838936, 556.538452, 269.009613, 0.334717, 0 1.334135,150.286057, 37.091347, 172.622589, 0.400635, 24 58.269234,143.487976, 104.038467, 172.501190, 0.308594, 24 163.076935,87.525238, 600.000000, 344.274017, 0.576660, 33 164.519241,84.975960, 320.000000, 344.516815, 0.389648, 33

Visualized result saved in ./infer_rknpu2.jpg

hedexiaodiao commented 8 months ago

找到原因了，看文档不仔细啊，实际安装版本是rknn_toolkit2-1.5.1b19，但是FastDeploy 默认librknnrt.so 库是1.4.2b3 这个版本统一起来就可以正常在3568上rknn 正常推理了，根据文档到网盘中下载和转换工具相同版本的librknnrt.so 图片

结果 root@RK356X:/userdata/app_data/pp/ppdete# ./infer_picodet_demo picodet_s_416_coco_lcnet 000000014439.jpg 1 [INFO] fastdeploy/vision/common/processors/transform.cc(45)::FuseNormalizeCast Normalize and Cast are fused to Normalize in preprocessing pipeline. [INFO] fastdeploy/vision/common/processors/transform.cc(93)::FuseNormalizeHWC2CHW Normalize and HWC2CHW are fused to NormalizeAndPermute in preprocessing pipeline. [INFO] fastdeploy/vision/common/processors/transform.cc(159)::FuseNormalizeColorConvert BGR2RGB and NormalizeAndPermute are fused to NormalizeAndPermute with swap_rb=1 [INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(81)::GetSDKAndDeviceVersion rknpu2 runtime version: 1.5.1b19 (32afb0e92@2023-07-14T12:46:17) [INFO] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(82)::GetSDKAndDeviceVersion rknpu2 driver version: 0.4.2 index=0, name=image, n_dims=4, dims=[1, 416, 416, 3], n_elems=519168, size=1038336, fmt=NHWC, type=FP16, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0 index=0, name=p2o.Mul.179, n_dims=4, dims=[1, 3598, 4, 1], n_elems=14392, size=28784, fmt=NCHW, type=FP32, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0 index=1, name=p2o.Concat.9, n_dims=4, dims=[1, 80, 3598, 1], n_elems=287840, size=575680, fmt=NCHW, type=FP32, qnt_type=AFFINE, zp=0, scale=1.000000, pass_through=0 [INFO] fastdeploy/runtime/runtime.cc(367)::CreateRKNPU2Backend Runtime initialized with Backend::RKNPU2 in Device::RKNPU. [INFO] fastdeploy/vision/common/processors/transform.cc(159)::FuseNormalizeColorConvert BGR2RGB and Normalize are fused to Normalize with swap_rb=1 [WARNING] fastdeploy/runtime/backends/rknpu2/rknpu2_backend.cc(420)::InitRKNNTensorMemory The input tensor type != model's inputs type.The input_type need FP16,but inputs[0].type is UINT8 [FastDeploy] PPDet in RKNPU2 duration = 0.131852s. DetectionResult: [xmin, ymin, xmax, ymax, score, label_id] 413.461548,89.649635, 508.461548, 282.605743, 0.833008, 0 160.961548,81.698311, 200.000000, 166.795670, 0.812012, 0 265.576935,79.756004, 299.230774, 167.524033, 0.791992, 0 105.384621,46.251198, 126.730774, 93.534248, 0.769531, 0 584.230774,113.685692, 612.692322, 178.085327, 0.762695, 0 328.269257,40.211838, 344.423096, 80.545067, 0.636719, 0 379.038483,42.093449, 396.153870, 83.397835, 0.554199, 0 510.000031,116.052879, 598.846191, 278.235565, 0.541504, 0 24.038462,116.417061, 55.048080, 153.442307, 0.443115, 0 58.557693,136.325714, 107.115387, 173.836533, 0.428955, 0 352.307709,45.097954, 376.923096, 104.034851, 0.421387, 0 188.750000,45.795971, 200.000000, 61.637917, 0.414551, 0 352.692322,45.097954, 369.615387, 87.343147, 0.388672, 0 360.961548,61.000599, 383.461548, 114.353363, 0.363525, 0 505.000031,114.838936, 556.538452, 269.009613, 0.334717, 0 1.334135,150.286057, 37.091347, 172.622589, 0.400635, 24 58.269234,143.487976, 104.038467, 172.501190, 0.308594, 24 163.076935,87.525238, 600.000000, 344.274017, 0.576660, 33 164.519241,84.975960, 320.000000, 344.516815, 0.389648, 33

Visualized result saved in ./infer_rknpu2.jpg

请问您是否在交叉编译RK3588环境时遇到了无法链接librknnrt.so的问题？请问如何解决呢？ 按照仓库文档里提供下载的gcc-linaro-6.3.1-2017.05-x86_64_aarch64-linux-gnu在x86 ubuntu20.04进行交叉编译时，会报错 /bin/ld: third_libs/install/rknpu2_runtime/lib/librknnrt.so: error adding symbols: file in wrong format collect2: error: ld returned 1 exit status

ltj19900609 commented 2 months ago

@erroot 你好，你有1.4.2那个版本的rk吗，那个网盘链接打不开了，能给我发一份吗

133673 commented 1 month ago

您好我下载最新的so之后并替换还是报这个错为什么呢 pi@NanoPC-T6-LTS:~/rk_zo/1.6/rknn_model_zoo-1.6.0/install/rk3588_linux_aarch64/rknn_yolov8_demo$ sudo ./rknn_yolov8_demo model/yolov8.rknn model/bus.jpg load lable ./model/coco_80_labels_list.txt E RKNN: [16:40:58.701] 6, 1 E RKNN: [16:40:58.701] Invalid RKNN model version 6 E RKNN: [16:40:58.701] rknn_init, load model failed! rknn_init fail! ret=-1 init_yolov8_model fail! ret=-1 model_path=model/yolov8.rknn Segmentation fault