An OCR Toolbox for Vietnamese Documents

This toolbox provides a pipeline to do OCR in Vietnamese documents (such as receipts, personal id, licenses,...). The project also support flexibility for adaptation.

:bookmark_tabs: More infomation:

• Report: link
• Youtube:

Pipeline in detail:

1. Use Canny Edge Detector and then detect contours.
2. Extract receipt from image and normalize.
3. Use Pixel Agreation Network (PAN) to detect text regions from extracted receipt, then crop these regions.
4. Use VietOCR to extract texts from regions, then perform word correction.
5. Retrieve information

Notebooks

• Notebook for training PAN:

• Notebook for training Transformer OCR:

• Notebook for training PhoBERT:

• Notebook for inference:

Pipeline

There are two stages (can also run in second stage only):

• The first stage is to detect and rectify document in the image, then forward through the "process flow" to find the best orientation of the document.
• The second stage is to forward the rotated image through the entire "process flow" normally to retrieve information

Datasets

• MCOCR-2020 (for detection)
• SROIE19 (for ocr and retrieval)

Pretrained weights

• Pretrained PAN weights on SROIE19:

• Pretrained OCR weights on MCOCR2021:

• Pretrained PhoBERT weights on MCOCR2021:

Inference

• Install dependencies pip install -r requirements.txt

• Full pipeline:

  python run.py --input=<input image> --output=<output folder>

  • Extra Parameters:
  • --debug: whether to save output of each step
  • --find_best_rotation: whether to find best rotation first
  • --do_retrieve: whether to retrieve information (based on class defined in config) or ocr only

References

• https://github.com/WenmuZhou/PAN.pytorch
• https://github.com/andrewdcampbell/OpenCV-Document-Scanner
• https://github.com/pbcquoc/vietocr