search

PaddlePaddle / Paddle-Lite

PaddlePaddle High Performance Deep Learning Inference Engine for Mobile and Edge (飞桨高性能深度学习端侧推理引擎）
https://www.paddlepaddle.org.cn/lite
Apache License 2.0
6.96k stars 1.61k forks source link

使用PaddleX可视化训练出的MobileNetV1量化版本，部署在paddlelite rknpu上，推理报错 #5129

Closed Z-Xiong closed 3 years ago

Z-Xiong commented 3 years ago

我使用paddlex训练了一版mobilenet1的简单分类模型，并使用客户端量化，同时使用opt工具转换为适用于‘rknpu和arm’的nb模型，将此模型替换掉paddle-lite的rknpu模型，推理结果出现如下错误：

[root@RV1126_RV1109:/userdata/bin]# ./image_classification_demo ./quant_model.nb
   ./synset_words.txt  ./tabby_cat.raw 
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by ./image_classification_demo)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by ./image_classification_demo)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by ./image_classification_demo)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by ./image_classification_demo)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by libpaddle_light_api_shared.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by libpaddle_light_api_shared.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by libpaddle_light_api_shared.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by libpaddle_light_api_shared.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by libpaddle_light_api_shared.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by libpaddle_light_api_shared.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by libpaddle_light_api_shared.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by libpaddle_light_api_shared.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by libpaddle_light_api_shared.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by libpaddle_light_api_shared.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by librknpu_ddk.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by librknpu_ddk.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by librknpu_ddk.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by librknpu_ddk.so)
[I  1/ 4 17: 7:10. 59 ...ang/Paddle-Lite/lite/core/device_info.cc:219 get_cpu_arch] Unknow cpu arch: 3079
[I  1/ 4 17: 7:10. 59 ...ang/Paddle-Lite/lite/core/device_info.cc:219 get_cpu_arch] Unknow cpu arch: 3079
[I  1/ 4 17: 7:10. 61 ...ang/Paddle-Lite/lite/core/device_info.cc:1064 Setup] ARM multiprocessors name: MODEL NAME      : ARMV7 PROCESSOR REV 5 (V7L)
HARDWARE        : GENERIC DT BASED SYSTEM

[I  1/ 4 17: 7:10. 61 ...ang/Paddle-Lite/lite/core/device_info.cc:1065 Setup] ARM multiprocessors number: 4
[I  1/ 4 17: 7:10. 61 ...ang/Paddle-Lite/lite/core/device_info.cc:1067 Setup] ARM multiprocessors ID: 0, max freq: 1512, min freq: 1512, cluster ID: 0, CPU ARCH: A-1
[I  1/ 4 17: 7:10. 61 ...ang/Paddle-Lite/lite/core/device_info.cc:1067 Setup] ARM multiprocessors ID: 1, max freq: 1512, min freq: 1512, cluster ID: 0, CPU ARCH: A-1
[I  1/ 4 17: 7:10. 61 ...ang/Paddle-Lite/lite/core/device_info.cc:1067 Setup] ARM multiprocessors ID: 2, max freq: 1512, min freq: 1512, cluster ID: 0, CPU ARCH: A-1
[I  1/ 4 17: 7:10. 61 ...ang/Paddle-Lite/lite/core/device_info.cc:1067 Setup] ARM multiprocessors ID: 3, max freq: 1512, min freq: 1512, cluster ID: 0, CPU ARCH: A-1
[I  1/ 4 17: 7:10. 61 ...ang/Paddle-Lite/lite/core/device_info.cc:1073 Setup] L1 DataCache size is: 
[I  1/ 4 17: 7:10. 61 ...ang/Paddle-Lite/lite/core/device_info.cc:1075 Setup] 32 KB
[I  1/ 4 17: 7:10. 61 ...ang/Paddle-Lite/lite/core/device_info.cc:1075 Setup] 32 KB
[I  1/ 4 17: 7:10. 61 ...ang/Paddle-Lite/lite/core/device_info.cc:1075 Setup] 32 KB
[I  1/ 4 17: 7:10. 61 ...ang/Paddle-Lite/lite/core/device_info.cc:1075 Setup] 32 KB
[I  1/ 4 17: 7:10. 61 ...ang/Paddle-Lite/lite/core/device_info.cc:1077 Setup] L2 Cache size is: 
[I  1/ 4 17: 7:10. 61 ...ang/Paddle-Lite/lite/core/device_info.cc:1079 Setup] 512 KB
[I  1/ 4 17: 7:10. 61 ...ang/Paddle-Lite/lite/core/device_info.cc:1079 Setup] 512 KB
[I  1/ 4 17: 7:10. 61 ...ang/Paddle-Lite/lite/core/device_info.cc:1079 Setup] 512 KB
[I  1/ 4 17: 7:10. 61 ...ang/Paddle-Lite/lite/core/device_info.cc:1079 Setup] 512 KB
[I  1/ 4 17: 7:10. 61 ...ang/Paddle-Lite/lite/core/device_info.cc:1081 Setup] L3 Cache size is: 
[I  1/ 4 17: 7:10. 61 ...ang/Paddle-Lite/lite/core/device_info.cc:1083 Setup] 0 KB
[I  1/ 4 17: 7:10. 61 ...ang/Paddle-Lite/lite/core/device_info.cc:1083 Setup] 0 KB
[I  1/ 4 17: 7:10. 61 ...ang/Paddle-Lite/lite/core/device_info.cc:1083 Setup] 0 KB
[I  1/ 4 17: 7:10. 61 ...ang/Paddle-Lite/lite/core/device_info.cc:1083 Setup] 0 KB
[I  1/ 4 17: 7:10. 61 ...ang/Paddle-Lite/lite/core/device_info.cc:1085 Setup] Total memory: 1024228KB
[I  1/ 4 17: 7:10. 62 ...ang/Paddle-Lite/lite/core/device_info.cc:447 check_cpu_online] CPU id:3 is offine
[W  1/ 4 17: 7:10. 62 ...ang/Paddle-Lite/lite/core/device_info.cc:1177 SetRunMode] Some cores are offline, switch to NO BIND MODE
[W  1/ 4 17: 7:10. 63 ...e-Lite/lite/model_parser/model_parser.cc:811 LoadModelNaiveFromFile] warning: the version of opt that transformed this model is not consistent with current Paddle-Lite version.
      version of opt:0d77fa3
      version of current Paddle-Lite:2708c2fe
[I  1/ 4 17: 7:10.239 ...e/lite/kernels/rknpu/subgraph_compute.cc:148 PrepareForRun] [RKNPU]:PrepareForRun
[I  1/ 4 17: 7:10.239 ...e/lite/kernels/rknpu/subgraph_compute.cc:160 Run] [RKNPU]:Run
[I  1/ 4 17: 7:10.239 ...e/lite/kernels/rknpu/subgraph_compute.cc:32 BuildDeviceProgram] [RKNPU]:BuildDeviceProgram
[F  1/ 4 17: 7:10.249 ...le-Lite/lite/model_parser/cpp/op_desc.cc:43 FindAttr] Check failed: it != desc.attrs().end(): No attributes called weight_scale found
[F  1/ 4 17: 7:10.249 ...le-Lite/lite/model_parser/cpp/op_desc.cc:43 FindAttr] Check failed: it != desc.attrs().end(): No attributes called weight_scale found

[F  1/ 4 17: 7:10.249 ...le-Lite/lite/model_parser/cpp/op_desc.cc:46 FindAttr] Check failed: attr_it != desc.attr_types().end(): 
[F  1/ 4 17: 7:10.249 ...le-Lite/lite/model_parser/cpp/op_desc.cc:46 FindAttr] Check failed: attr_it != desc.attr_types().end(): 

[F  1/ 4 17: 7:10.249 ...le-Lite/lite/model_parser/cpp/op_desc.cc:114 GetAttr<std::vector<float> >] Check failed: pair.second->second == AttrType::FLOATS: required type is FLOATS not match the true type
[F  1/ 4 17: 7:10.249 ...le-Lite/lite/model_parser/cpp/op_desc.cc:114 GetAttr<std::vector<float> >] Check failed: pair.second->second == AttrType::FLOATS: required type is FLOATS not match the true type

[F  1/ 4 17: 7:10.249 ...xiongzhuang/Paddle-Lite/lite/utils/any.h:62 get] Check failed: type_ == typeid(T).hash_code(): 
[F  1/ 4 17: 7:10.249 ...xiongzhuang/Paddle-Lite/lite/utils/any.h:62 get] Check failed: type_ == typeid(T).hash_code(): 

Segmentation fault (core dumped)

请问问题出现在哪里呢？

juncaipeng commented 3 years ago

RK上量化的模型要求：使用quant-post-static量化方法，权重使用abs_max量化方式，模型必须是全量化（is_full_quantize设置为True），需要提前融合conv+bn（optimize_model设置为True）。目前RK上无法做到支持任意模型，WIP。

PaddleSlim的api文档：https://paddleslim.readthedocs.io/zh_CN/latest/api_cn/quantization_api.html#quant-post-static

Z-Xiong commented 3 years ago

需要提前融合conv+bn（optimize_model设置为True）

paddleslim 1.2以上版本支持optimize_model参数

juncaipeng commented 3 years ago

嗯，是的，具体参考paddleslim哦。

Z-Xiong commented 3 years ago

@juncaipeng 按照您的提示我将量化参数设置如下：

    quant_post_static(
        executor=exe,
        model_dir=args.model_path,
        quantize_model_path=args.save_path,
        sample_generator=val_reader,
        model_filename=args.model_filename,
        params_filename=args.params_filename,
        batch_size=args.batch_size,
        batch_nums=args.batch_num,
        is_full_quantize=True,
        weight_quantize_type='abs_max',
        optimize_model=True)

可以量化导出模型，但是在量化时会出现如下报警：

2021-01-11 17:22:32,798-INFO: Collect quantized variable names ...
2021-01-11 17:22:32,799-WARNING: feed is not supported for quantization.
2021-01-11 17:22:32,800-WARNING: scale is not supported for quantization.
2021-01-11 17:22:32,800-WARNING: fetch is not supported for quantization.

然后使用opt工具进行模型转换： ./opt --model_file=inference_model_quant/__model__ --param_file=inference_model_quant/__params__ --optimize_out_type=naive_buffer --optimize_out=rknpu --valid_targets=rknpu,arm

已经使用opt工具查看支持该模型MobileNetV1

最后在rknpu预测时还是出现如下错误：

[root@RV1126_RV1109:/userdata/bin]# ./image_classification_demo ./rknpu.nb   ./s
ynset_words.txt  ./tabby_cat.raw 
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by ./image_classification_demo)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by ./image_classification_demo)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by ./image_classification_demo)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by ./image_classification_demo)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by libpaddle_light_api_shared.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by libpaddle_light_api_shared.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by libpaddle_light_api_shared.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by libpaddle_light_api_shared.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by libpaddle_light_api_shared.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by libpaddle_light_api_shared.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by libpaddle_light_api_shared.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by libpaddle_light_api_shared.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by libpaddle_light_api_shared.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by libpaddle_light_api_shared.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by librknpu_ddk.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by librknpu_ddk.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by librknpu_ddk.so)
./image_classification_demo: /lib/libstdc++.so.6: no version information available (required by librknpu_ddk.so)
[I  1/11 17: 2: 5.681 ...ang/Paddle-Lite/lite/core/device_info.cc:219 get_cpu_arch] Unknow cpu arch: 3079
[I  1/11 17: 2: 5.681 ...ang/Paddle-Lite/lite/core/device_info.cc:219 get_cpu_arch] Unknow cpu arch: 3079
[I  1/11 17: 2: 5.683 ...ang/Paddle-Lite/lite/core/device_info.cc:1064 Setup] ARM multiprocessors name: MODEL NAME  : ARMV7 PROCESSOR REV 5 (V7L)
HARDWARE    : GENERIC DT BASED SYSTEM

[I  1/11 17: 2: 5.683 ...ang/Paddle-Lite/lite/core/device_info.cc:1065 Setup] ARM multiprocessors number: 4
[I  1/11 17: 2: 5.683 ...ang/Paddle-Lite/lite/core/device_info.cc:1067 Setup] ARM multiprocessors ID: 0, max freq: 1512, min freq: 1512, cluster ID: 0, CPU ARCH: A-1
[I  1/11 17: 2: 5.683 ...ang/Paddle-Lite/lite/core/device_info.cc:1067 Setup] ARM multiprocessors ID: 1, max freq: 1512, min freq: 1512, cluster ID: 0, CPU ARCH: A-1
[I  1/11 17: 2: 5.683 ...ang/Paddle-Lite/lite/core/device_info.cc:1067 Setup] ARM multiprocessors ID: 2, max freq: 1512, min freq: 1512, cluster ID: 0, CPU ARCH: A-1
[I  1/11 17: 2: 5.683 ...ang/Paddle-Lite/lite/core/device_info.cc:1067 Setup] ARM multiprocessors ID: 3, max freq: 1512, min freq: 1512, cluster ID: 0, CPU ARCH: A-1
[I  1/11 17: 2: 5.683 ...ang/Paddle-Lite/lite/core/device_info.cc:1073 Setup] L1 DataCache size is: 
[I  1/11 17: 2: 5.684 ...ang/Paddle-Lite/lite/core/device_info.cc:1075 Setup] 32 KB
[I  1/11 17: 2: 5.684 ...ang/Paddle-Lite/lite/core/device_info.cc:1075 Setup] 32 KB
[I  1/11 17: 2: 5.684 ...ang/Paddle-Lite/lite/core/device_info.cc:1075 Setup] 32 KB
[I  1/11 17: 2: 5.684 ...ang/Paddle-Lite/lite/core/device_info.cc:1075 Setup] 32 KB
[I  1/11 17: 2: 5.684 ...ang/Paddle-Lite/lite/core/device_info.cc:1077 Setup] L2 Cache size is: 
[I  1/11 17: 2: 5.684 ...ang/Paddle-Lite/lite/core/device_info.cc:1079 Setup] 512 KB
[I  1/11 17: 2: 5.684 ...ang/Paddle-Lite/lite/core/device_info.cc:1079 Setup] 512 KB
[I  1/11 17: 2: 5.684 ...ang/Paddle-Lite/lite/core/device_info.cc:1079 Setup] 512 KB
[I  1/11 17: 2: 5.684 ...ang/Paddle-Lite/lite/core/device_info.cc:1079 Setup] 512 KB
[I  1/11 17: 2: 5.684 ...ang/Paddle-Lite/lite/core/device_info.cc:1081 Setup] L3 Cache size is: 
[I  1/11 17: 2: 5.684 ...ang/Paddle-Lite/lite/core/device_info.cc:1083 Setup] 0 KB
[I  1/11 17: 2: 5.684 ...ang/Paddle-Lite/lite/core/device_info.cc:1083 Setup] 0 KB
[I  1/11 17: 2: 5.684 ...ang/Paddle-Lite/lite/core/device_info.cc:1083 Setup] 0 KB
[I  1/11 17: 2: 5.684 ...ang/Paddle-Lite/lite/core/device_info.cc:1083 Setup] 0 KB
[I  1/11 17: 2: 5.684 ...ang/Paddle-Lite/lite/core/device_info.cc:1085 Setup] Total memory: 1024228KB
[I  1/11 17: 2: 5.685 ...ang/Paddle-Lite/lite/core/device_info.cc:447 check_cpu_online] CPU id:3 is offine
[W  1/11 17: 2: 5.685 ...ang/Paddle-Lite/lite/core/device_info.cc:1177 SetRunMode] Some cores are offline, switch to NO BIND MODE
[W  1/11 17: 2: 5.686 ...e-Lite/lite/model_parser/model_parser.cc:811 LoadModelNaiveFromFile] warning: the version of opt that transformed this model is not consistent with current Paddle-Lite version.
      version of opt:0d77fa3
      version of current Paddle-Lite:2708c2fe
[I  1/11 17: 2: 5.893 ...e/lite/kernels/rknpu/subgraph_compute.cc:148 PrepareForRun] [RKNPU]:PrepareForRun
[I  1/11 17: 2: 5.893 ...e/lite/kernels/rknpu/subgraph_compute.cc:160 Run] [RKNPU]:Run
[I  1/11 17: 2: 5.896 ...e/lite/kernels/rknpu/subgraph_compute.cc:32 BuildDeviceProgram] [RKNPU]:BuildDeviceProgram
[F  1/11 17: 2: 5.908 ...le-Lite/lite/model_parser/cpp/op_desc.cc:43 FindAttr] Check failed: it != desc.attrs().end(): No attributes called weight_scale found
[F  1/11 17: 2: 5.908 ...le-Lite/lite/model_parser/cpp/op_desc.cc:43 FindAttr] Check failed: it != desc.attrs().end(): No attributes called weight_scale found

[F  1/11 17: 2: 5.908 ...le-Lite/lite/model_parser/cpp/op_desc.cc:46 FindAttr] Check failed: attr_it != desc.attr_types().end(): 
[F  1/11 17: 2: 5.908 ...le-Lite/lite/model_parser/cpp/op_desc.cc:46 FindAttr] Check failed: attr_it != desc.attr_types().end(): 

[F  1/11 17: 2: 5.908 ...le-Lite/lite/model_parser/cpp/op_desc.cc:114 GetAttr<std::vector<float> >] Check failed: pair.second->second == AttrType::FLOATS: required type is FLOATS not match the true type
[F  1/11 17: 2: 5.908 ...le-Lite/lite/model_parser/cpp/op_desc.cc:114 GetAttr<std::vector<float> >] Check failed: pair.second->second == AttrType::FLOATS: required type is FLOATS not match the true type

[F  1/11 17: 2: 5.908 ...xiongzhuang/Paddle-Lite/lite/utils/any.h:62 get] Check failed: type_ == typeid(T).hash_code(): 
[F  1/11 17: 2: 5.908 ...xiongzhuang/Paddle-Lite/lite/utils/any.h:62 get] Check failed: type_ == typeid(T).hash_code(): 

Segmentation fault (core dumped)

请问可能原因是什么呢？是否是量化时的警告信息引起的？

juncaipeng commented 3 years ago

产出量化模型应该没问题了，warning没关系。

你可以对比一下你的量化模型和demo中提供的量化模型差异，可能paddlex训练的有差异。

paddlelite提供的demo可以执行吗？

Z-Xiong commented 3 years ago

@juncaipeng paddlelite提供的demo里的模型是可以在rknpu上运行的，替换为我自己训练的模型就会报上述错误。如何对比量化后的模型呢？nb模型似乎不可以用netron可视化阿

juncaipeng commented 3 years ago

这里提供了产出量化模型的demo：https://paddle-lite.readthedocs.io/zh/release-v2.8/demo_guides/rockchip_npu.html

建议不要用paddlex吧