Port the latex_envs jupyter extension to cocalc

williamstein commented 4 years ago

See https://pypi.org/project/jupyter_latex_envs/

We could probably make this generic and work for more general markdown in cocalc - not just jupyter! There is really nothing jupyter specific about this. It also seems reasonably backwards compat.

williamstein commented 4 years ago

I'm looking into this a little more carefully and unfortunately I'm less optimistic about how difficult this is. The reasons are that:

Looking at the page on latex_envs, it doesn't really say exactly what it does. It justs lists some ambiguous bullet points.
Looking at actual source code in latex_envs, I see things like this:
```
...
        data = data.replace('\\\\', '!sl!!sl!')
        data = re.sub(r'%([\S\s ]*?)\n', r'!cc!\1!nl!',data)
        #data = data.replace('%', '!cc!')
        data = data.replace("/begin", "\\begin")
        data = data.replace("/end", "\\end")
....
```
and I have no clue why or what is going on. It certainly seems replacing "/begin" by "\begin" is going to result in accidentally breaking a lot of normal things, i.e., this is a hacky ill-defined extension that you would only want to use when you explicitly need it. When you need it, then it is probably very useful.

williamstein commented 2 years ago

I did some more research on latex_envs. I read through some of the testing/example docs, and it seems like most of what latex_envs does is basically take latex that current markdown parsers in Jupyter view as "broken latex formulas" and interpret it using latex. For example:

\begin{itemize}
\item Hello
\item World
\end{itemize}

gets considered as latex via jupyter due to the \begin/\end tokens. Then, since of course mathjax knows nothing about this non-math construction, it's an error. This addresses one of my biggest concerns about latex_envs, namely that it would involve adding a lot of incompatibilities to parsing markdown. However, that's not the case at all -- instead, it's mostly just making it so the current parsing mechanism works in more generality.

Interestingly, this latex.js project I mentioned recently

https://latex.js.org/playground.html

is a pure frontend Javascript program for basically taking snippets of latex and rendering them nicely as html.It's also very extensible (for adding new environments, tracking state, etc.). Thus it seems to me like a goodway to proceed is for me to simply extend the rendering of "math" in CoCalc to use both mathjax and latexjs together. The result is just that you can suddenly put chunks of latex into notebooks. An additional step is to manage global state so that cross references (\label/\ref) have the right scope, and also to implement the most popular environments (e.g., latexjs doesn't implement a theorem environment by default).

Conceptually, I think this is very similar to how Markdown embeds arbitrary blocks of HTML, but with the\HTML language directly replaced by "latex".

Finally, it should certainly be possible to write something that converts all use of this sort of latex in markdown just to normal markdown + html. That could be used to export a version of all notebooks that can be viewed in all other Jupyter clients, thus addressing my worry about the broader ecosystem.

williamstein commented 1 month ago

I've decided to definitely not port this in general, as there is a lot of potential weirdness. I have already done a limited ported \label, \ref, itemize, enumerate and figures though, but ONLY in Jupyter notebooks (not any markdown file).

sagemathinc / cocalc

Port the latex_envs jupyter extension to cocalc #4896