编译安装paddle-custom-npu-0.0.0后使用paddleocr特别慢

问题描述 / Problem Description

运行环境 / Runtime Environment

OS: Ubuntu 22.04
Paddle: paddle-custom-npu-0.0.0
PaddleOCR: 2.7.3

复现代码 / Reproduction Code

from paddleocr import PaddleOCR

def load_pp_ocr(): ocr_loader = PaddleOCR( use_angle_cls=True, lang="ch", use_npu=True, det_limit_side_len=680 ) return ocr_loader

def pp_ocr_box(ocr_loader: PaddleOCR, img_path): result = ocr_loader.ocr(img_path) if result: for one_box in result[0]: [lb, rb, rt, lt], (text, det_threshold) = one_box yield (lb, rb, rt, lt, text, det_threshold)

def main(): ocr_loader = load_pp_ocr() print("load finish\n") filePath = "test.png" for box in pp_ocr_box(ocr_loader, filePath): if box: lb, rb, rt, lt, text, det_threshold = box print(lb, rb, rt, lt, text, det_threshold) print("test.png is ok\n") if name == "main": main()

完整报错 / Complete Error Message

基础功能检查没有问题。但是在paddleocr预测时：

I0531 15:46:17.706218 112503 init.cc:236] ENV [CUSTOM_DEVICE_ROOT]=/usr/local/lib/python3.10/dist-packages/paddle_custom_device I0531 15:46:17.706283 112503 init.cc:145] Try loading custom device libs from: [/usr/local/lib/python3.10/dist-packages/paddle_custom_device] I0531 15:46:18.313715 112503 custom_device.cc:1099] Succeed in loading custom runtime in lib: /usr/local/lib/python3.10/dist-packages/paddle_custom_device/libpaddle-custom-npu.so I0531 15:46:18.319205 112503 custom_kernel.cc:63] Succeed in loading 350 custom kernel(s) from loaded lib(s), will be used like native ones. I0531 15:46:18.319399 112503 init.cc:157] Finished in LoadCustomDevice with libs_path: [/usr/local/lib/python3.10/dist-packages/paddle_custom_device] I0531 15:46:18.319448 112503 init.cc:242] CustomDevice: npu, visible devices count: 8 [2024/05/31 15:46:19] ppocr DEBUG: Namespace(help='==SUPPRESS==', use_gpu=False, use_xpu=False, use_npu=True, ir_optim=True, use_tensorrt=False, min_subgraph_size=15, precision='fp32', gpu_mem=500, gpu_id='0,1,2,3', image_dir=None, page_num=0, det_algorithm='DB', det_model_dir='/root/.paddleocr/whl/det/ch/ch_PP-OCRv4_det_infer', det_limit_side_len=680, det_limit_type='max', det_box_type='quad', det_db_thresh=0.3, det_db_box_thresh=0.6, det_db_unclip_ratio=1.5, max_batch_size=10, use_dilation=False, det_db_score_mode='fast', det_east_score_thresh=0.8, det_east_cover_thresh=0.1, det_east_nms_thresh=0.2, det_sast_score_thresh=0.5, det_sast_nms_thresh=0.2, det_pse_thresh=0, det_pse_box_thresh=0.85, det_pse_min_area=16, det_pse_scale=1, scales=[8, 16, 32], alpha=1.0, beta=1.0, fourier_degree=5, rec_algorithm='SVTR_LCNet', rec_model_dir='/root/.paddleocr/whl/rec/ch/ch_PP-OCRv4_rec_infer', rec_image_inverse=True, rec_image_shape='3, 48, 320', rec_batch_num=6, max_text_length=25, rec_char_dict_path='/usr/local/lib/python3.10/dist-packages/paddleocr/ppocr/utils/ppocr_keys_v1.txt', use_space_char=True, vis_font_path='./doc/fonts/simfang.ttf', drop_score=0.5, e2e_algorithm='PGNet', e2e_model_dir=None, e2e_limit_side_len=768, e2e_limit_type='max', e2e_pgnet_score_thresh=0.5, e2e_char_dict_path='./ppocr/utils/ic15_dict.txt', e2e_pgnet_valid_set='totaltext', e2e_pgnet_mode='fast', use_angle_cls=True, cls_model_dir='/root/.paddleocr/whl/cls/ch_ppocr_mobile_v2.0_cls_infer', cls_image_shape='3, 48, 192', label_list=['0', '180'], cls_batch_num=6, cls_thresh=0.9, enable_mkldnn=False, cpu_threads=10, use_pdserving=False, warmup=False, sr_model_dir=None, sr_image_shape='3, 32, 128', sr_batch_num=1, draw_img_save_dir='./inference_results', save_crop_res=False, crop_res_save_dir='./output', use_mp=True, total_process_num=1, process_id=0, benchmark=True, save_log_path='./log_output/', show_log=True, use_onnx=False, output='./output', table_max_len=488, table_algorithm='TableAttn', table_model_dir=None, merge_no_span_structure=True, table_char_dict_path=None, layout_model_dir=None, layout_dict_path=None, layout_score_threshold=0.5, layout_nms_threshold=0.5, kie_algorithm='LayoutXLM', ser_model_dir=None, re_model_dir=None, use_visual_backbone=True, ser_dict_path='../train_data/XFUND/class_list_xfun.txt', ocr_order_method=None, mode='structure', image_orientation=False, layout=True, table=True, ocr=True, recovery=False, use_pdf2docx_api=False, invert=False, binarize=False, alphacolor=(255, 255, 255), lang='ch', det=True, rec=True, type='ocr', ocr_version='PP-OCRv4', structure_version='PP-StructureV2') load finish

I0531 15:46:21.097870 113010 init.cc:236] ENV [CUSTOM_DEVICE_ROOT]=/usr/local/lib/python3.10/dist-packages/paddle_custom_device I0531 15:46:21.097935 113010 init.cc:145] Try loading custom device libs from: [/usr/local/lib/python3.10/dist-packages/paddle_custom_device] I0531 15:46:21.287110 113251 init.cc:236] ENV [CUSTOM_DEVICE_ROOT]=/usr/local/lib/python3.10/dist-packages/paddle_custom_device I0531 15:46:21.287169 113251 init.cc:145] Try loading custom device libs from: [/usr/local/lib/python3.10/dist-packages/paddle_custom_device] I0531 15:46:21.682519 113010 custom_device.cc:1099] Succeed in loading custom runtime in lib: /usr/local/lib/python3.10/dist-packages/paddle_custom_device/libpaddle-custom-npu.so I0531 15:46:21.687526 113010 custom_kernel.cc:63] Succeed in loading 350 custom kernel(s) from loaded lib(s), will be used like native ones. I0531 15:46:21.687721 113010 init.cc:157] Finished in LoadCustomDevice with libs_path: [/usr/local/lib/python3.10/dist-packages/paddle_custom_device] I0531 15:46:21.687773 113010 init.cc:242] CustomDevice: npu, visible devices count: 8 I0531 15:46:21.876283 113251 custom_device.cc:1099] Succeed in loading custom runtime in lib: /usr/local/lib/python3.10/dist-packages/paddle_custom_device/libpaddle-custom-npu.so I0531 15:46:21.879951 113251 custom_kernel.cc:63] Succeed in loading 350 custom kernel(s) from loaded lib(s), will be used like native ones. I0531 15:46:21.880151 113251 init.cc:157] Finished in LoadCustomDevice with libs_path: [/usr/local/lib/python3.10/dist-packages/paddle_custom_device] I0531 15:46:21.880192 113251 init.cc:242] CustomDevice: npu, visible devices count: 8

[2024/05/31 15:49:47] ppocr DEBUG: dt_boxes num : 3, elapsed : 32.64580488204956 [2024/05/31 15:49:47] ppocr DEBUG: cls num : 3, elapsed : 0.022808074951171875 Warning: tiling offset out of range, index: 32 Warning: tiling offset out of range, index: 32 Warning: tiling offset out of range, index: 32 [2024/05/31 15:50:49] ppocr DEBUG: rec_res num : 3, elapsed : 62.09084606170654 [8.0, 7.0] [752.0, 7.0] [752.0, 25.0] [8.0, 25.0] 如果想要使用om模型进行推理，就需要使用华为acl的特定的Api，关于atc的api具体可查看他们 0.9951098561286926 [9.0, 35.0] [742.0, 35.0] [742.0, 53.0] [9.0, 53.0] 的官网的api说明，其实如果说学过CUDA的话，就会发现，它的Api和CUDA其实蛮像的，比如 0.9868237972259521 [8.0, 62.0] [331.0, 61.0] [331.0, 80.0] [8.0, 81.0] cudamemorymalloc对应acl.rt.malloc等 0.9853638410568237 ###############ocr_test1.png is ok########### [2024/05/31 15:51:24] ppocr DEBUG: dt_boxes num : 6, elapsed : 35.48471665382385 [2024/05/31 15:51:55] ppocr DEBUG: cls num : 6, elapsed : 30.283159494400024 Warning: tiling offset out of range, index: 32 Warning: tiling offset out of range, index: 32 Warning: tiling offset out of range, index: 32 [2024/05/31 15:53:00] ppocr DEBUG: rec_res num : 6, elapsed : 65.58153176307678 [32.0, 9.0] [379.0, 9.0] [379.0, 24.0] [32.0, 24.0] 1.npu加载.om模型，并将模型转移到NPU上 0.9712381362915039 [31.0, 38.0] [279.0, 38.0] [279.0, 53.0] [31.0, 53.0] 2.NPU上分配模型的输入和输出 0.9975005388259888 [29.0, 64.0] [278.0, 64.0] [278.0, 83.0] [29.0, 83.0] 3.CPU加载图片，从硬盘到内存 0.9993540644645691 [29.0, 91.0] [280.0, 91.0] [280.0, 110.0] [29.0, 110.0] 4.输入数据拷贝，从内存到NPU 0.9980849027633667 [29.0, 120.0] [158.0, 120.0] [158.0, 139.0] [29.0, 139.0] 5.NPU执行推理 0.9916012287139893 [30.0, 149.0] [280.0, 149.0] [280.0, 164.0] [30.0, 164.0] 6.输出数据拷贝，从NPU到内存 0.9904201030731201 ###############ocr_test2.png is ok###########

预测的时间都在一分钟以上，而且NPU 能跑满60G（这个很奇怪），

PaddlePaddle / PaddleOCR