Introduction

MindOCR is an open-source toolbox for OCR development and application based on MindSpore, which integrates series of mainstream text detection and recognition algorihtms/models, provides easy-to-use training and inference tools. It can accelerate the process of developing and deploying SoTA text detection and recognition models in real-world applications, such as DBNet/DBNet++ and CRNN/SVTR, and help fulfill the need of image-text understanding.

Installation

Quick Start

1. Text Detection and Recognition Demo

After installing MindOCR, we can run text detection and recognition on an arbitrary image easily as follows.

python tools/infer/text/predict_system.py --image_dir {path_to_img or dir_to_imgs} \
                                          --det_algorithm DB++  \
                                          --rec_algorithm CRNN

After running, the results will be saved in ./inference_results by default. Here is an example result.

Visualization of text detection and recognition result

We can see that all texts on the image are detected and recognized accurately. For more usage, please refer to the inference section in tutorials.

2. Model Training and Evaluation - Quick Guideline

It is easy to train your OCR model with the tools/train.py script, which supports both text detection and recognition model training.

python tools/train.py --config {path/to/model_config.yaml}

The --config arg specifies the path to a yaml file that defines the model to be trained and the training strategy including data process pipeline, optimizer, lr scheduler, etc.

MindOCR provides SoTA OCR models with their training strategies in configs folder. You may adapt it to your task/dataset, for example, by running

# train text detection model DBNet++ on icdar15 dataset
python tools/train.py --config configs/det/dbnet/dbpp_r50_icdar15.yaml

# train text recognition model CRNN on icdar15 dataset
python tools/train.py --config configs/rec/crnn/crnn_icdar15.yaml

Similarly, it is easy to evaluate the trained model with the tools/eval.py script.

python tools/eval.py \
    --config {path/to/model_config.yaml} \
    --opt eval.dataset_root={path/to/your_dataset} eval.ckpt_load_path={path/to/ckpt_file}

For more illustration and usage, please refer to the model training section in Tutorials.

3. Model Offline Inference - Quick Guideline

You can do MindSpore Lite inference in MindOCR using MindOCR models or Third-party models (PaddleOCR, MMOCR, etc.). Please refer to the following documents

• Python/C++ Inference on Ascend 310
• MindOCR Models Offline Inference - Quick Start
• Third-party Models Offline Inference - Quick Start.

Tutorials

• Datasets
  • Dataset Preparation
  • Data Transformation Mechanism
• Model Training
  • Yaml Configuration
  • Text Detection
  • Text Recognition
  • Distributed Training
  • Advance: Gradient Accumulation, EMA, Resume Training, etc
• Inference with MindSpore
  • Python Online Inference
• Inference with MindSpore Lite
  • Python/C++ Inference on Ascend 310
  • MindOCR Models Offline Inference - Quick Start
  • Third-party Models Offline Inference - Quick Start
  • Model Conversion
• Developer Guides
  • Customize Dataset
  • Customize Data Transformation
  • Customize a New Model
  • Customize Postprocessing Method

Model List

For the detailed performance of the trained models, please refer to configs.

For details of MindSpore Lite and ACL inference models support, please refer to MindOCR Models Support List and Third-party Models Support List (PaddleOCR, MMOCR, etc.).

Dataset List

MindOCR provides a dataset conversion tool to OCR datasets with different formats and support customized dataset by users. We have validated the following public OCR datasets in model training/evaluation.

We will include more datasets for training and evaluation. This list will be continuously updated.

Frequently Asked Questions

Frequently asked questions about configuring environment and mindocr, please refer to FAQ.

Notes

What is New

How to Contribute

We appreciate all kinds of contributions including issues and PRs to make MindOCR better.

Please refer to CONTRIBUTING.md for the contributing guideline. Please follow the Model Template and Guideline for contributing a model that fits the overall interface :)

License

This project follows the Apache License 2.0 open-source license.

Citation

If you find this project useful in your research, please consider citing:

@misc{MindSpore OCR 2023,
    title={{MindSpore OCR }:MindSpore OCR Toolbox},
    author={MindSpore Team},
    howpublished = {\url{https://github.com/mindspore-lab/mindocr/}},
    year={2023}
}