使用PaddleLite转.nb模型并部署到Android端后精度下降

lexiaoyuan commented 2 years ago

基于PaddleOCR项目，使用自定义数据集和字典训练的识别模型，inference模型识别效果很好，参考文档在Linux环境编译Paddle-Lite得到预测库和opt工具，使用opt工具将inference模型转为.nb模型后，在android_demo中使用，发现识别效果变差。同一张图，同一个字典，inference模型识别效果： Android端识别效果：

paddle-bot-old[bot] commented 2 years ago

您好，我们已经收到了您的问题，会安排技术人员尽快解答您的问题，请耐心等待。请您再次检查是否提供了清晰的问题描述、复现代码、环境&版本、报错信息等。同时，您也可以通过查看官网文档、常见问题、历史Issue来寻求解答。祝您生活愉快～

Hi! We've received your issue and please be patient to get responded. We will arrange technicians to answer your questions as soon as possible. Please make sure that you have posted enough message to demo your request. You may also check out the API，FAQ and Github Issue to get the answer.Have a nice day!

lexiaoyuan commented 2 years ago

这个问题目前是解决了（解决详情参考https://github.com/PaddlePaddle/PaddleOCR/issues/3876#issuecomment-953445141），但是，是通过使用PaddleOCR中的原始字典，而没有用自定义的字典，仅仅做了这一个改变。所以我推测会不会是使用自定义字典训练的识别模型，通过Paddle Lite转模型后，精度会下降。究竟是因为使用了自定义的字典训练的原因（但是推理模型表现良好啊），还是使用Paddle Lite转模型出现问题（但是转换使用原始字典ppocr/utils/ppocr_keys_v1.txt训练的模型就没有问题啊），目前确实还没有确定。

LDOUBLEV commented 2 years ago

所以我推测会不会是使用自定义字典训练的识别模型，通过Paddle Lite转模型后，精度会下降。究竟是因为使用了自定义的字典训练的原因（但是推理模型表现良好啊），还是使用Paddle Lite转模型出现问题

使用android的时候是否修改了 android demo里的默认字典？

如果原模型转nb模型没问题，更换字典重新训练模型，再转nb模型部署时注意修改字典；

lexiaoyuan commented 2 years ago

所以我推测会不会是使用自定义字典训练的识别模型，通过Paddle Lite转模型后，精度会下降。究竟是因为使用了自定义的字典训练的原因（但是推理模型表现良好啊），还是使用Paddle Lite转模型出现问题

使用android的时候是否修改了 android demo里的默认字典？

修改过，预处理和后处理的参数都检查过，字典都是一样的。字典不一样的话，识别结果的那个索引取出来的会是汉字。

LDOUBLEV commented 2 years ago

有用这个demo调试过吗： https://github.com/PaddlePaddle/PaddleOCR/tree/release%2F2.3/deploy/lite

android demo是基于这个做的，如果lite demo换字典后运行没问题的话，可以先排除模型的问题

lexiaoyuan commented 2 years ago

有用这个demo调试过吗： https://github.com/PaddlePaddle/PaddleOCR/tree/release%2F2.3/deploy/lite

android demo是基于这个做的，如果lite demo换字典后运行没问题的话，可以先排除模型的问题

10张图片测试结果和在安卓端的效果一样。 (没有方向分类，相关配置已改过。这里就给出一张图片的测试结果)

用自定义字典训练的模型：测试结果：(识别错误)

umi:/data/local/tmp/debug $ ./ocr_db_crnn det_db_watermeter_opt.nb  rec_number_watermeter_opt.nb  ch_ppocr_mobile_v2.0_cls_slim_opt.nb  ./22.jpg  water_meter_number_dict.txt
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1097 Setup] ARM multiprocessors name: HARDWARE        : QUALCOMM TECHNOLOGIES, INC SM8250
_UMI_KONA_
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1098 Setup] ARM multiprocessors number: 8
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1100 Setup] ARM multiprocessors ID: 0, max freq: 1804, min freq: 1804, cluster ID: 1, CPU ARCH: A55
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1100 Setup] ARM multiprocessors ID: 1, max freq: 1804, min freq: 1804, cluster ID: 1, CPU ARCH: A55
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1100 Setup] ARM multiprocessors ID: 2, max freq: 1804, min freq: 1804, cluster ID: 1, CPU ARCH: A55
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1100 Setup] ARM multiprocessors ID: 3, max freq: 1804, min freq: 1804, cluster ID: 1, CPU ARCH: A55
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1100 Setup] ARM multiprocessors ID: 4, max freq: 2419, min freq: 2419, cluster ID: 0, CPU ARCH: A77
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1100 Setup] ARM multiprocessors ID: 5, max freq: 2419, min freq: 2419, cluster ID: 0, CPU ARCH: A77
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1100 Setup] ARM multiprocessors ID: 6, max freq: 2419, min freq: 2419, cluster ID: 0, CPU ARCH: A77
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1100 Setup] ARM multiprocessors ID: 7, max freq: 2841, min freq: 2841, cluster ID: 0, CPU ARCH: A77
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1106 Setup] L1 DataCache size is:
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1108 Setup] 256 KB
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1108 Setup] 256 KB
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1108 Setup] 256 KB
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1108 Setup] 256 KB
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1108 Setup] 192 KB
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1108 Setup] 192 KB
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1108 Setup] 192 KB
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1108 Setup] 192 KB
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1110 Setup] L2 Cache size is:
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1112 Setup] 512 KB
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1112 Setup] 512 KB
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1112 Setup] 512 KB
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1112 Setup] 512 KB
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1112 Setup] 768 KB
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1112 Setup] 768 KB
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1112 Setup] 768 KB
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1112 Setup] 768 KB
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1114 Setup] L3 Cache size is:
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1116 Setup] 4096 KB
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1116 Setup] 4096 KB
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1116 Setup] 4096 KB
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1116 Setup] 4096 KB
[I 11/19 21: 5:45.335 ...ite/Paddle-Lite/lite/core/device_info.cc:1116 Setup] 4096 KB
[I 11/19 21: 5:45.336 ...ite/Paddle-Lite/lite/core/device_info.cc:1116 Setup] 4096 KB
[I 11/19 21: 5:45.336 ...ite/Paddle-Lite/lite/core/device_info.cc:1116 Setup] 4096 KB
[I 11/19 21: 5:45.336 ...ite/Paddle-Lite/lite/core/device_info.cc:1116 Setup] 4096 KB
[I 11/19 21: 5:45.336 ...ite/Paddle-Lite/lite/core/device_info.cc:1118 Setup] Total memory: 7824076KB
The detection visualized image saved in ./vis.jpg
0       80      0.999235
花费了0.830565秒

用原始字典训练的模型识别结果：（识别正确）

umi:/data/local/tmp/debug $ ./ocr_db_crnn det_db_watermeter_opt.nb  rec_number_watermeter_ch_opt.nb  ch_ppocr_mobile_v2.0_cls_slim_opt.nb  ./22.jpg  ppocr_keys_v1.txt
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1097 Setup] ARM multiprocessors name: HARDWARE        : QUALCOMM TECHNOLOGIES, INC SM8250
_UMI_KONA_
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1098 Setup] ARM multiprocessors number: 8
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1100 Setup] ARM multiprocessors ID: 0, max freq: 1804, min freq: 1804, cluster ID: 1, CPU ARCH: A55
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1100 Setup] ARM multiprocessors ID: 1, max freq: 1804, min freq: 1804, cluster ID: 1, CPU ARCH: A55
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1100 Setup] ARM multiprocessors ID: 2, max freq: 1804, min freq: 1804, cluster ID: 1, CPU ARCH: A55
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1100 Setup] ARM multiprocessors ID: 3, max freq: 1804, min freq: 1804, cluster ID: 1, CPU ARCH: A55
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1100 Setup] ARM multiprocessors ID: 4, max freq: 2419, min freq: 2419, cluster ID: 0, CPU ARCH: A77
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1100 Setup] ARM multiprocessors ID: 5, max freq: 2419, min freq: 2419, cluster ID: 0, CPU ARCH: A77
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1100 Setup] ARM multiprocessors ID: 6, max freq: 2419, min freq: 2419, cluster ID: 0, CPU ARCH: A77
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1100 Setup] ARM multiprocessors ID: 7, max freq: 2841, min freq: 2841, cluster ID: 0, CPU ARCH: A77
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1106 Setup] L1 DataCache size is:
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1108 Setup] 256 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1108 Setup] 256 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1108 Setup] 256 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1108 Setup] 256 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1108 Setup] 192 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1108 Setup] 192 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1108 Setup] 192 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1108 Setup] 192 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1110 Setup] L2 Cache size is:
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1112 Setup] 512 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1112 Setup] 512 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1112 Setup] 512 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1112 Setup] 512 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1112 Setup] 768 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1112 Setup] 768 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1112 Setup] 768 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1112 Setup] 768 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1114 Setup] L3 Cache size is:
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1116 Setup] 4096 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1116 Setup] 4096 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1116 Setup] 4096 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1116 Setup] 4096 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1116 Setup] 4096 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1116 Setup] 4096 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1116 Setup] 4096 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1116 Setup] 4096 KB
[I 11/19 21: 1:54.535 ...ite/Paddle-Lite/lite/core/device_info.cc:1118 Setup] Total memory: 7824076KB
The detection visualized image saved in ./vis.jpg
0       80178   0.999804
花费了0.832401秒

lexiaoyuan commented 2 years ago

补充：使用自定义字典训练的模型，在Android demo中，lite demo中pdserving(详情见https://github.com/PaddlePaddle/Serving/issues/1530)中，10张图片的输出都是一样的，（只识别了前一两个字符）。

dium6i commented 2 years ago

遇到了同样的问题，换用自己的字典训练的模型转nb后在Android平台识别效果很差，使用inference模型就一点问题都没有，如下图1，图2。图1 Android平台识别结果： 2 10识别结果

图2 使用inference模型识别结果：服务器识别结果

图3 修改predict_rec.py前的inference模型识别结果：服务器修改

换用自己的字典的模型在刚训练好的时候，出现了如下 issue 中提到的模型转inference前后，识别结果有差异的问题，见上图3。 https://github.com/PaddlePaddle/PaddleOCR/issues/3758

根据官方建议，将下列代码注释

https://github.com/PaddlePaddle/PaddleOCR/blob/ebd266656b344b16e38a3299e3e77014aba337de/tools/infer/predict_rec.py#L89-L90

之后问题得以解决。有没有可能在Android Demo的代码里也有类似的数据预处理不一致问题呢？@LDOUBLEV

PaddlePaddle / Paddle-Lite

使用PaddleLite转.nb模型并部署到Android端后精度下降 #7676