UserWangZz
commented
3 weeks ago Fine tuning the RE model cannot improve the accuracy of text detection and recognition, but it will enhance the ability to determine text relationships. If you have made minor adjustments to the custom data, you can replace the official OCR model during the inference phase. The specific command to run is as follows:
python3 ./tools/infer_kie_token_ser_re.py \ -c configs/kie/vi_layoutxlm/re_vi_layoutxlm_xfund_zh.yml \ -o Architecture.Backbone.checkpoints=./pretrained_model/re_vi_layoutxlm_xfund_pretrained/best_accuracy \ Global.infer_img=./train_data/XFUND/zh_val/image/zh_val_42.jpg \ Global.kie_det_model_dir=path/to/your/det_model \ Global.kie_rec_model_dir=path/to/your/rec_model \ -c_ser configs/kie/vi_layoutxlm/ser_vi_layoutxlm_xfund_zh.yml \ -o_ser Architecture.Backbone.checkpoints=./pretrained_model/ser_vi_layoutxlm_xfund_pretrained/best_accuracy
For more specific steps, please refer to the documentation: https://github.com/PaddlePaddle/PaddleOCR/blob/main/ppstructure/kie/README_ch.md
piarosebelledelapaz
i did an inference using the pretrained model for paddleOCR KIE for relation extraction, but i noticed that it is not detecting all the objects in the image, and the text being recognized is also not accurate.
if i finetune the model for RE using my own dataset, would this also increase the accuracy for text detection and recognition or just determining the relationship between two words?
i already finetuned the object detection model and text recognition model of PaddleOCR using my custom dataset, would the fine-tuned weights be useful in a way for training the RE model?