gpu测试和cpu测试结果不一样

[wangning7149]

请问，同一个推理模型，gpu测试和cpu测试结果不一样 ，为什么呢？怎么解决？

[littletomatodonkey]

把具体的配置都说一下吧，配置的超参之类的

[zjykzj]

把具体的配置都说一下吧，配置的超参之类的

你好 @littletomatodonkey

部署

我们使用了默认配置的工程参数。具体模型如下：

1. 检测：ch_det_r50_vd_db
2. 识别：ch_rec_r34_vd_crnn_enhance

部署方式是使用已集成的HubServing，参考服务部署

• 设置环境变量

$ export PYTHONPATH=/home/zj/xxx/

• CPU启动:

$ l$ hub serving start --modules ocr_system_xxx --port 9076 --use_multiprocess --workers
ocr_system_xxx == 1.0.0
[2020-09-09 12:19:02 +0800] [2125] [INFO] Starting gunicorn 20.0.4
[2020-09-09 12:19:02 +0800] [2125] [INFO] Listening at: http://0.0.0.0:9076 (2125)
[2020-09-09 12:19:02 +0800] [2125] [INFO] Using worker: sync
[2020-09-09 12:19:02 +0800] [3090] [INFO] Booting worker with pid: 3090

GPU启动：

$ CUDA_VISIBLE_DEVICES=3 hub serving start -c deploy/ocr_system_xxx/config.json 
use gpu:  True
CUDA_VISIBLE_DEVICES:  3
ocr_system_expressbill == 1.0.0
 * Serving Flask app "paddlehub.serving.app_single" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
2020-09-09 13:22:08,049-INFO:  * Running on http://0.0.0.0:18088/ (Press CTRL+C to quit)

测试

CPU测试：

$ python test/deploy/test_hubserving.py http://127.0.0.1:9076/predict/ocr_system_expressbill test/deploy/src/
2020-09-09 13:20:20,318-INFO: Predict time of test/deploy/src/WeChat Image_20200828092006.png: 3.289s
2020-09-09 13:20:20,318-INFO: {'number': '18216840871'}
2020-09-09 13:20:21,254-INFO: Predict time of test/deploy/src/WeChat Image_20200828092031.png: 0.915s
2020-09-09 13:20:21,254-INFO: []
2020-09-09 13:20:23,152-INFO: Predict time of test/deploy/src/WeChat Image_20200828092014.png: 1.887s
2020-09-09 13:20:23,152-INFO: []
2020-09-09 13:20:24,435-INFO: Predict time of test/deploy/src/WeChat Image_20200828092049.png: 1.283s
2020-09-09 13:20:24,435-INFO: {'number': '15585839650'}
2020-09-09 13:20:24,817-INFO: Predict time of test/deploy/src/WeChat Image_20200828092019.png: 0.381s
2020-09-09 13:20:24,817-INFO: []
2020-09-09 13:20:27,579-INFO: Predict time of test/deploy/src/WeChat Image_20200828092038.png: 2.761s
2020-09-09 13:20:27,579-INFO: {'number': '18285888669'}
2020-09-09 13:20:27,579-INFO: avg time cost: 1.7526456514994304

GPU测试：

$ python test/deploy/test_hubserving.py http://127.0.0.1:18088/predict/ocr_system_expressbill test/deploy/src/ 
2020-09-09 13:23:04,077-INFO: Predict time of test/deploy/src/WeChat Image_20200828092006.png: 6.968s
2020-09-09 13:23:04,077-INFO: {'number': '18216840871'}
2020-09-09 13:23:04,161-INFO: Predict time of test/deploy/src/WeChat Image_20200828092031.png: 0.084s
2020-09-09 13:23:04,161-INFO: {'number': '15585839650'}
2020-09-09 13:23:04,225-INFO: Predict time of test/deploy/src/WeChat Image_20200828092014.png: 0.063s
2020-09-09 13:23:04,225-INFO: {'number': '15186268886'}
2020-09-09 13:23:04,283-INFO: Predict time of test/deploy/src/WeChat Image_20200828092049.png: 0.057s
2020-09-09 13:23:04,283-INFO: {'number': '15585839650'}
2020-09-09 13:23:04,351-INFO: Predict time of test/deploy/src/WeChat Image_20200828092019.png: 0.068s
2020-09-09 13:23:04,352-INFO: []
2020-09-09 13:23:04,440-INFO: Predict time of test/deploy/src/WeChat Image_20200828092038.png: 0.089s
2020-09-09 13:23:04,441-INFO: {'number': '18285888669'}
2020-09-09 13:23:04,441-INFO: avg time cost: 1.2214858929316204

[littletomatodonkey]

可能是cpu下多张图片预测时，内存异常的问题，可以拉下最新代码试下，同时关闭mkldnn，mkldnn的问题会在Paddle2.0正式版统一修复