certik / theoretical-physics

Source code of the Theoretical Physics Reference online book
https://theoretical-physics.com
MIT License
222 stars 48 forks source link

Use sphinx-math-dollar #94

Open asmeurer opened 5 years ago

asmeurer commented 5 years ago

It would be great if you could try out using https://www.sympy.org/sphinx-math-dollar/ instead of https://github.com/certik/theoretical-physics/blob/master/exts/math_dollar.py, and let me know if you have an issues with it. You'll have to inspect the math manually to make sure it doesn't do anything wrong, although you can also build with MATH_DOLLAR_DEBUG=1, which will print messages about any $'s that are skipped (e.g., because they are in a code block).

certik commented 5 years ago

Can you summarize the differences between the two math_dollar implementations?

asmeurer commented 5 years ago

sphinx-math-dollar is more rigorous about not parsing dollars that shouldn't be parsed. For instance, if you have a doctest like

>>> print("$x$")
$x$

then sphinx-math-dollar won't parse it, because it ignores docstrings (this was important for SymPy because of the latex() docstring). It also ignores things in code (double backticks) and other things like that. The way it works is that it uses an AST transformer on the docutils tree, rather than replacing stuff in the raw rst text. So it can tell the difference between dollars in a paragraph and dollars in a code block. The debug environment variable I mentioned will print out any instances of dollars that are not parsed as math, so you can see if there are any.

I also improved the regular expression a little bit to handle some cases the old one wasn't handling correctly. It now no longer incorrectly parses something like

$ cd doc
$ make html

as math, because it ignores it if there is a space after the dollar. There were also some incorrect parsings of escaped dollars, like $13\$ + 14\$$ (I doubt you have any escaped dollars here, but it's worth mentioning).

Eventually I want to add support for double dollars for display math, so you don't have to use .. math:: anymore. I just need to modify the regular expression to parse them, or figure out some better way to lex dollars. If you have any suggestions on a good way to write a simple lexer in Python that can handle backslash escapes, let me know. Currently the regular expression with backtracking to handle the escapes is a bit unwieldy, and it will only get worse if we add double dollar support.

And finally, a lot of other people have shown interest in sphinx-math-dollar (I got a ton of feedback on Twitter). So hopefully it should get some use, and be well maintained. I put it under the sympy user on GitHub rather than my personal user so that we can maintain it as a community.

asmeurer commented 5 years ago

Also your extension doesn't work with docstrings, only stuff in rst files.

certik commented 5 years ago

I see, that sounds nice. I'll try it out.