identify symbols in Latex expressions

[bhpayne]

Separate a Latex-as-string into tokens.

See tests/valid_latex_expressions for examples of valid Latex expressions.

examples:

• a = b has tokens a, b, =
• r_{\rm Earth} = x^2 has tokens r_{\rm Earth}, x, =, ^, 2

The purpose of this issue is to help the user identify which symbols should be associated with the expression.

[msgoff]

Here is a sample taken from the git@github.com:allofphysicsgraph/latex2sympyparser repo The test expression for tokenization is in the file test.tex

to run docker run -it --rm latex:latest /bin/bash

rm tf_idf 
rm offsets 
./scanner.out .
python read_offsets.py 

for the equation \begin{equation}
u=\frac{-y}{x^2+y^2} , v=\frac{x}{x^2+y^2} , \text{and} w=0 \end{equation}