Latin recognition model outputs chinese characters

leihuayi commented 4 months ago

问题描述 / Problem Description

When I use the recognition model specialized in latin, I get an output in full chinese whereas the baseline model chinese/ch_PP-OCRv3_rec_infer.tar recognizes correctly the latin chars

Example on this image gi-pietta

WRONG output using model multilingual/latin_PP-OCRv3_rec_infer.tar

CORRECT output using model chinese/ch_PP-OCRv3_rec_infer.tar

Of course I tested on other images and I have the same bug.

运行环境 / Runtime Environment

OS: Debian 12 (Docker image 3.11-slim-bookworm)
Paddle: 2.6.1
PaddleOCR: 2.8.1

复现代码 / Reproduction Code

Download models in advance

curl -o det_infer.tar https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_det_infer.tar
curl -o rec_infer.tar https://paddleocr.bj.bcebos.com/PP-OCRv3/multilingual/latin_PP-OCRv3_rec_infer.tar
tar xf det_infer.tar -C /app/models/detection --strip-components=1 && tar xf rec_infer.tar -C /app/models/recognition --strip-components=1

python code

from paddleocr import PaddleOCR

OCRCustom=PaddleOCR(
    det_model_dir='/app/models/detection',
    rec_model_dir='/app/models/recognition',
    cls_model_dir='/app/models/recognition',
    use_angle_cls=False
)

....
result = OCRCustom.ocr(img_array)

完整报错 / Complete Error Message

See Problem Description

可能解决方案 / Possible solutions

?? probably the model in https://paddleocr.bj.bcebos.com/PP-OCRv3/multilingual/latin_PP-OCRv3_rec_infer.tar is NOT the latin model and a mistake was made. Maybe the person who developped this model should double-check it ?

附件 / Appendix

GreatV commented 4 months ago

I can't reproduce it.

from paddleocr import PaddleOCR, draw_ocr
from PIL import Image

ocr = PaddleOCR(use_angle_cls=True, lang="latin", 
det_model_dir="./pretrained_models/Multilingual_PP-OCRv3_det_infer",
rec_model_dir="./pretrained_models/latin_PP-OCRv3_rec_infer",
rec_char_dict_path="./ppocr/utils/dict/latin_dict.txt",
ocr_version="PP-OCRv3")
img_path = './350030224-92e69229-b59a-441f-9794-67b63763469a.jpeg'
result = ocr.ocr(img_path, cls=True)
for idx in range(len(result)):
    res = result[idx]
    for line in res:
        print(line)

result = result[0]
image = Image.open(img_path).convert('RGB')
boxes = [line[0] for line in result]
txts = [line[1][0] for line in result]
scores = [line[1][1] for line in result]
im_show = draw_ocr(image, boxes, txts, scores, font_path='./doc/fonts/simfang.ttf')
im_show = Image.fromarray(im_show)
im_show.save('result.jpg')

python test.py 
[2024/07/19 00:35:53] ppocr DEBUG: Namespace(help='==SUPPRESS==', use_gpu=True, use_xpu=False, use_npu=False, use_mlu=False, ir_optim=True, use_tensorrt=False, min_subgraph_size=15, precision='fp32', gpu_mem=500, gpu_id=0, image_dir=None, page_num=0, det_algorithm='DB', det_model_dir='./pretrained_models/Multilingual_PP-OCRv3_det_infer', det_limit_side_len=960, det_limit_type='max', det_box_type='quad', det_db_thresh=0.3, det_db_box_thresh=0.6, det_db_unclip_ratio=1.5, max_batch_size=10, use_dilation=False, det_db_score_mode='fast', det_east_score_thresh=0.8, det_east_cover_thresh=0.1, det_east_nms_thresh=0.2, det_sast_score_thresh=0.5, det_sast_nms_thresh=0.2, det_pse_thresh=0, det_pse_box_thresh=0.85, det_pse_min_area=16, det_pse_scale=1, scales=[8, 16, 32], alpha=1.0, beta=1.0, fourier_degree=5, rec_algorithm='SVTR_LCNet', rec_model_dir='./pretrained_models/latin_PP-OCRv3_rec_infer', rec_image_inverse=True, rec_image_shape='3, 48, 320', rec_batch_num=6, max_text_length=25, rec_char_dict_path='./ppocr/utils/dict/latin_dict.txt', use_space_char=True, vis_font_path='./doc/fonts/simfang.ttf', drop_score=0.5, e2e_algorithm='PGNet', e2e_model_dir=None, e2e_limit_side_len=768, e2e_limit_type='max', e2e_pgnet_score_thresh=0.5, e2e_char_dict_path='./ppocr/utils/ic15_dict.txt', e2e_pgnet_valid_set='totaltext', e2e_pgnet_mode='fast', use_angle_cls=True, cls_model_dir='/home/greatx/.paddleocr/whl/cls/ch_ppocr_mobile_v2.0_cls_infer', cls_image_shape='3, 48, 192', label_list=['0', '180'], cls_batch_num=6, cls_thresh=0.9, enable_mkldnn=False, cpu_threads=10, use_pdserving=False, warmup=False, sr_model_dir=None, sr_image_shape='3, 32, 128', sr_batch_num=1, draw_img_save_dir='./inference_results', save_crop_res=False, crop_res_save_dir='./output', use_mp=False, total_process_num=1, process_id=0, benchmark=False, save_log_path='./log_output/', show_log=True, use_onnx=False, return_word_box=False, output='./output', table_max_len=488, table_algorithm='TableAttn', table_model_dir=None, merge_no_span_structure=True, table_char_dict_path=None, layout_model_dir=None, layout_dict_path=None, layout_score_threshold=0.5, layout_nms_threshold=0.5, kie_algorithm='LayoutXLM', ser_model_dir=None, re_model_dir=None, use_visual_backbone=True, ser_dict_path='../train_data/XFUND/class_list_xfun.txt', ocr_order_method=None, mode='structure', image_orientation=False, layout=True, table=True, ocr=True, recovery=False, use_pdf2docx_api=False, invert=False, binarize=False, alphacolor=(255, 255, 255), lang='latin', det=True, rec=True, type='ocr', savefile=False, ocr_version='PP-OCRv3', structure_version='PP-StructureV2')
[2024/07/19 00:35:53] ppocr WARNING: The first GPU is used for inference by default, GPU ID: 0
[2024/07/19 00:35:53] ppocr WARNING: The first GPU is used for inference by default, GPU ID: 0
[2024/07/19 00:35:53] ppocr WARNING: The first GPU is used for inference by default, GPU ID: 0
[2024/07/19 00:35:54] ppocr DEBUG: dt_boxes num : 3, elapsed : 0.24906659126281738
[2024/07/19 00:35:54] ppocr DEBUG: cls num  : 3, elapsed : 0.0252535343170166
[2024/07/19 00:35:54] ppocr DEBUG: rec_res num  : 3, elapsed : 0.03034210205078125
[[[527.0, 1027.0], [624.0, 1027.0], [624.0, 1058.0], [527.0, 1058.0]], ('August', 0.9970551133155823)]
[[[624.0, 1025.0], [669.0, 1025.0], [669.0, 1056.0], [624.0, 1056.0]], ('29,', 0.9808254837989807)]
[[[677.0, 1026.0], [748.0, 1026.0], [748.0, 1055.0], [677.0, 1055.0]], ('2022', 0.9992855787277222)]

result

leihuayi commented 4 months ago

Hello, You are right I missed something in my code, I did not add the parameter lang="latin", at initialization of PaddleOCR model. I did not see it was necessary though it was written here https://github.com/PaddlePaddle/PaddleOCR/blob/main/doc/doc_en/multi_languages_en.md

As improvement, maybe it would be better to setup the multi-language models so that their default language is the same as their name ? (for instance, latin language model = latin setup)

PaddlePaddle / PaddleOCR