MathReader: API for handwritten mathematical expressions recognition

Paper: MathReader: API for handwritten mathematical expressions recognition, published in 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI)

Details about implementation are described in the paper.

Interface for validation:

GitHub Project: MathReader Validation

MathReader Neural Network Training

If you want to re-train the neural network model. Feel free to improve it (details about the training process is described in the paper mentioned above).

GitHub Project: MathReader Training

About:

• Accepted symbols:

  • {"0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "a", "b", "c", "m", "n", "x", "y", "z", "+", "-", "*", "sqrt", "=", "neq"}

• Internal modules:

  • Image preprocessing
  • Image posprocessing
  • Symbol recognition
  • Structural analysis
  • Lexical analysis
  • Sintatic analysis
  • Grammar correction

Usage:

Example in: main.py

from api import *

hme_recognizer = HME_Recognizer()

image = ['images/numbers/teste10.png']

try:
    hme_recognizer.load_image(image)
    expression, img = hme_recognizer.recognize()
    print("Latex: ", expression)
except Exception as e:
    print('Exception: ', e)

Configuration

Configuration file is in docs/config.json

{
    "application": {
        "debug_mode": "active",
        "debug_mode_image": "active"
    }
}

• debug_mode: active activate debug messages in terminal
• debug_mode_image: active show the images during the execution for debugging

HME_Recognizer

All attributes and methods avaiable in __HME_Recognizer__ class at api.py

Attributes:

- image
- parsed_expression
- processed_image
- predictions
- configurations
- expression_after_recognition
- expression_after_parser
- expression_after_grammar
- parser_tree
- parser_list
- lex_errors
- yacc_errors
- pure_lex_errors
- pure_yacc_errors

Methods

• load_image(image)

  • First method you should call.
  • It loads the image containing the expression.
  • Input: image.
  • Returns: None.

• recognize()

  • Second method you should call.
  • It makes the recognition of the image containing the expression.
  • Input: None.
  • Returns:
    • expression: latex of the expression
    • image: image after processing

• reset()

  • Reset all attributes from HME_Recognizer()
  • Input: None.
  • Returns: None

• get_predictions()

  • Get all predictions from neural network
  • Input: None.
  • Returns: List of predictions.

• get_expression_after_parser()

  • Get expression after going to parser module
  • Input: None.
  • Returns: (list) expression

• get_expression_after_grammar()

  • Get expression after grammar check and correction
  • Input: None.
  • Returns: (str) expression

• get_expression_after_recognition()

  • Get expression after recognition module
  • Input: None.
  • Returns: (dict) expression

• get_configurations()

  • Get configuration from configuration file docs/config.json
  • Input: None.
  • Returns: configurations

• get_lex_errors()

  • Get lex errors with a structure used by the grammar check

• get_yacc_errors()

  • Get yacc errors with a structure used by the grammar check

• get_pure_lex_errors()

  • Get 'pure' lex errors coming from PLY (Python Lex Yacc)

• get_pure_yacc_errors()

  • Get 'pure' yacc errors coming from PLY (Python Lex Yacc)