search

mindspore-lab / mindocr

A toolbox of ocr models and algorithms based on MindSpore
https://mindspore-lab.github.io/mindocr/
Apache License 2.0
230 stars 56 forks source link

[Model] CAN Model Support #731

Open zhangjunlongtech opened 3 months ago

zhangjunlongtech commented 3 months ago

  1. Background MindOCR does not currently support handwritten mathematical formula recognition. I hope to solve this problem by contributing a CAN model.

  2. Model Introduction HMER(Handwritten Mathematical Expression Recognition) mostly uses the encoder-decoder mechanism. However, when identifying long or complex formulas, it cannot guarantee the accuracy of the region of interest of the Attention module. CAN(Counting-Aware Network) utilizes Multi-Scale Counting Module to improve the accuracy of formula recognition, by introducing counting vectors that can provide global information and spatial position codes that can provide position information.

  3. Reference Papers When Counting Meets HMER: Counting-Aware Network for Handwritten Mathematical Expression Recognition Bohan Li, Ye Yuan, Dingkang Liang, Xiao Liu, Zhilong Ji, Jinfeng Bai, Wenyu Liu, Xiang Bai ECCV, 2022

  4. Expectations I'd like to get the community's comments, suggestions, and related reference documents or PRs.

kk928290341 commented 3 months ago

Contributions are welcome, and they are greatly appreciated! You can refer to the CONTRIBUTING documentation, and please provide the test report and README when submitting the PR.