github / markup

Determines which markup library to use to render a content file (e.g. README) on GitHub
MIT License
5.87k stars 3.39k forks source link

Rendering math equations? #897

Closed IreneCrisologo closed 7 years ago

IreneCrisologo commented 8 years ago

Hi folks, Any updates on this? Rendering math on README pages would be very very helpful.

I know it can be done using online tools like iTex2Img but it would be easier if the normal latex expressions can be digested.

Thanks a bunch! -i

adavidzh commented 8 years ago

Also #274 was asking to go in this direction. No chance to have a supported way to render math in github README files?

jayvdb commented 8 years ago

I've been doing a little analysis on https://github.com/imsky/PocketEuler/issues/44

One simple and secure approach is to use the small mimetex (code mirrorred at https://github.com/icaoberg/mimetex) , which is a standalone renderrer, not using any other larger binaries like latex.

https://packages.debian.org/search?suite=jessie&searchon=names&keywords=mimetex https://admin.fedoraproject.org/pkgdb/package/rpms/mimetex/

No CVE since 20090713, which a batch of problems was found. See also https://www.debian.org/security/2009/dsa-1917.en.html

Another approach is the use the MediaWiki approach, which is also battle tested, of texvccheck/texvc , an OCaml program , which has a small checker program (texvccheck) that checks the AMS-Latex before sending to a second program that using latex to render it.

0 CVE : https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=texvc

jayvdb commented 8 years ago

@bkeepers , you closed #274 saying "can't be resolved by a change to the code in this project".

So, which project(s) should this feature be added to? Each of the processors? Or a new project to do that math rendering only?

Most people are interested in maths in markdown , which is https://github.com/vmg/redcarpet , but they have rejected that feature request : https://github.com/vmg/redcarpet/issues/313 and a smaller version of it https://github.com/vmg/redcarpet/issues/395

RST and docutils supports maths, but this has been disabled https://github.com/github/markup/issues/83

asciidoctor supports MathJax (https://github.com/asciidoctor/asciidoctor/issues/492) , but without MathJax JS loaded by github markup it wont work. And there is https://github.com/asciidoctor/asciidoctor-latex

mediawiki has <math> and wikicloth has supported it since 2011 (https://github.com/nricciar/wikicloth/issues/19 / https://github.com/nricciar/wikicloth/blob/master/lib/wikicloth/extensions/math.rb) , but again it isnt enabled by github markup.

Is there a plan? What is the plan? What do you need? How can we help?

jayvdb commented 8 years ago

Note related request for MathML : https://github.com/github/markup/issues/551

jayvdb commented 8 years ago

Maybe a first step is for Github to treat typical math mode as non-markdown mode, similar to backticks (..).

Currently Github renders

Let $\text{S}_1(N) = \sum_{p=1}^N \text{E}(p)$

with the _ denoting italics, as

"Let $\text{S}1(N) = \sum{p=1}^N \text{E}(p)$"

The same thing happens with *.

AaronNGray commented 8 years ago

@bkeepers I have found a stop gap solution a site https://www.codecogs.com/latex/eqneditor.php that generates embedded links with TeX editor that then embeds a gif in GitHub :-

double click on the equation to be taken to the editor ;)

adavidzh commented 8 years ago

Hey @AaronNGray, that is pretty nifty!

adavidzh commented 8 years ago

I'm wondering if the codecogs workaround of @AaronNGray could be packaged with github.io+MathJax?

AaronNGray commented 8 years ago

It would be very easy to make a similar webapp to codecogs https://www.codecogs.com/latex/eqneditor.php. This would give the security isolation that GitHub needs by having it as a separate web based application.

It would be useful to attach persistant URL's to equations and use them for accessing the images and the editor rather than the long URL's codecogs uses.

Also codecogs does not support many packages. It does not support the 'semantic' and 'maths-symbols' that I need in order to make this useful and I am sure others will want other packages.

https://www.ctan.org/tex-archive/macros/latex/contrib/semantic https://www.ctan.org/pkg/maths-symbols

The following should come out as an inference rule :-

and codecogs does not support \usepackage{}

AaronNGray commented 8 years ago

I am putting together a Node.js solution for this.

taiya commented 8 years ago

I taught that GitHub originally had MathJax on his pages, later on removed it to make sure everything was nice and responsive. Why would it be so difficult to have a "preview" button that enables mathjax on demand? Wikipedia seems to be doing math, yet maintain responsiveness, just fine!

FlorianFranzen commented 8 years ago

+1

allefeld commented 8 years ago

How about opt-in MathJax on a per-repository basis? General GitHub responsiveness would not be impaired, but simple math would be there for people who need it.

taiya commented 8 years ago

@allefeld anything really, but removing it cold-turkey creates a big problem. Basically any mathematical software cannot use the wiki to document the software :(

AaronNGray commented 8 years ago

I will see if I can get a third party TeX based solution in place ASAP. I got distracted trying to get Angular2 working in order to do the UI.

jayvdb commented 8 years ago

A workaround mentioned by @cyhsutw at https://github.com/education/classroom/issues/675#issuecomment-245474372 :

Another hack is utilizing the Jupyter Notebook.

GitHub has built-in support for rendering .ipynb files. You can write inline and display LaTeX code in the notebook and GitHub will render it for you.

Here's a sample notebook file: https://gist.github.com/cyhsutw/d5983d166fb70ff651f027b2aa56ee4e

ycpei commented 8 years ago

+1 to this. I am thinking of hosting maths manuscripts on Github, where issues can be discussions about the papers.

I'm OK with certain walkarounds mentioned here as long as they work. However in general maths people are not fans of hacky solutions, and without native maths rendering mathematicians are much less likely to use Github.

Also, can anyone explain why mathoverflow can do it but not Github?

allefeld commented 8 years ago

@bkeepers is this something we can reasonably expect to be handled at some point, or does it fall in the category nevergonnahappen? Just wondering whether to stay subscribed to this issue.

taiya commented 8 years ago

I guess the question falls in this category: "Can github classroom (partially) override the GitHub wiki system?"

By the way, this is how I did it: http://gfx.uvic.ca/teaching/dgp/hw1.md

(sorry, I am part both of the classroom thread and the classic github thread)

allefeld commented 8 years ago

@ataiya no, the original question is about README files in regular repositories. The classroom question may be related, but this is about markup rendering on github in general. Hence the repository it was issued to.

AaronNGray commented 8 years ago

The technique to do this I simple just an embedding of a an image reference in to a external site that does TeX rendering to images. I outlined earlier https://github.com/github/markup/issues/897#issuecomment-231591884 and had meant to put something in place by now. I have done a local PoC using node.js but have got vastly distracted by 101 other problems that needed solving.

Anyone do node.js and/or Angular 2 can you contact me to work on this please ?

taiya commented 8 years ago

having to embed images would be terrible, as you'd lose the ability to use the Markdown editor to change equations and you'd have to rely on something that might not be any longer supported in the future (external website)

AaronNGray commented 8 years ago

My solution would store the TeX formulae on the separate site or as Gists or in your repo and render to images that you can click on in the rendered GitHub markdown file taking you to the external site for editing of the TeX equation which updates the image. You could have the top 20, 50 or 100 public equations on the sites home page and support public and private and shared libraries of formulae.

gschenk commented 7 years ago

The most important advantage of markdown, that it can be read in source as well as rendered, is unfortunately lost in the gif approach @AaronNGray.

@jayvdb had a very good point in an earlier comment. Even if the maths are not rendered, it would be very helpful to treat something delimited by $...$ and $$...$$ as literals. I think most people who have anything whatsoever to do with computer sciences or science are fine with deciphering latex.

@bkeepers You've mentioned two years ago that KaTeX was still in its infancy and thus not suitable. It seems they had a couple hundred commits since. Would you mind to comment on it in the light of its recent state? (KaTeX live demo)

dmyersturnbull commented 7 years ago

I'd like to add supplement the point @ycpei made because this issue is so important to us. We already use LaTeX extensively in our Issues and Markdown files to discuss algorithms and math. There's no good alternative, but LaTeX is admittedly hard for some users to read. This feature would be extremely helpful to us and other computational scientists.

jan-matthis commented 7 years ago

If you are using Chrome or Firefox, this may help: https://github.com/FlorianFranzen/KaTeXify

It's an unpacked WebExtension loading KaTeX (plus its auto-renderer) on GitHub.

dmyersturnbull commented 7 years ago

@jan-matthis Awesome work! It seems to work, mostly (I'll file a bug report).

eggie5 commented 7 years ago

I vote for the notebook syntax style of $$..$$ and $...$ for inline...

AaronNGray commented 7 years ago

How does KaTex work I cannot find either documentation or a proper code base ?

IreneCrisologo commented 7 years ago

@AaronNGray the codecogs solution is indeed a nice and quick workaround for simple equations (thanks for that suggestion!), but as you have mentioned it is very limited. Like @ycpei I am also using Github to host math-heavy documents written in markdown for discussions with colleagues.

IreneCrisologo commented 7 years ago

I just saw this other workaround: readme2tex

According to the README page:

readme2tex is a Python script that "texifies" your readme. It takes in Github Markdown and replaces anything enclosed between dollar signs with rendered LaTeX

What it does is that it parses the markdown file, reads the equation, renders it in rawgit.com as an image, and links it back to the markdown file as an embedded image.

The good thing about it is that it can be done automatically (according to the author), but what I don't like about it is that it replaces the equation completely.

To illustrate what I mean, here is what happens: This: screen shot 2017-02-18 at 12 30 51

Becomes this: screen shot 2017-02-18 at 12 31 57

jayvdb commented 7 years ago

Something like KaTeXify doesnt help here ... GitHub mangles the latex because it parses it as markdown. At the very least, GitHub needs to treat it as a literal.

taiya commented 7 years ago

I would like to stress this is not something we'd be looking for workarounds for. This is something so central to software development (its math backend) that I think it is simply embarrassing that github doesn't offer any support for it!!!

analog-cbarber commented 7 years ago

Note that GitLab now supports KaTeX based math: https://docs.gitlab.com/ee/user/markdown.html#math

taiya commented 7 years ago

But GitLab is a completely different product, so how's that related? Unless you want to give the github people a wakeup call

analog-cbarber commented 7 years ago

It's related because people that really care about this feature might want to know that depending on their situation might be able to just switch to GitLab.

AaronNGray commented 7 years ago

This will only be up on and off for a while until I get it done properly as tex.tools up an online

http://aarongray.org

but it creates PNG's for TeX and allows them to be served. Cut and paste the into a markdown file.

nothing is persistent yet so created tests may disappear !

test :-

https://github.com/AaronNGray/test

click on the image to edit it.

AaronNGray commented 7 years ago

Ooh, someone managed to evade my blocklists with some fake SPAM :) I will add proper user accounts when I get round to it, this is only a PoC at the mo.

casprwang commented 7 years ago

Upvote for built-in support, workarounds are terrible at this moment.

AaronNGray commented 7 years ago

I just need to get GitHub API sussed for both private and public accounts and hopefully will end up with something near enough seamless.

dexhunter commented 7 years ago

If anyone is using chrome, this extension(Github with MathJax) should be useful. :)

jrappen commented 7 years ago

Until this issue is fixed I suggest people use Github Pages (repo settings) with ./docs in master branch in combination with:

For your index.html use something similar to:

Click here for example code ```html Description
Loading...
```


The best part about this:

jan-matthis commented 7 years ago

GH pages might be an alternative, but the content will be public.

Regarding the browser extension approach (see previous post), there is a fundamental limitation when using markdown: Asterisks and underscores inside formulae may get replaced for <em> tags which messes the rendering up. Since both are used emphasis, it's not possible to correct for it.

https://github.github.com/gfm/#emphasis-and-strong-emphasis

kivikakk commented 7 years ago

👋 Hi there! Since this issue doesn't relate to the github-markup gem itself, I'm closing it. I see a lot of people opening and commenting on issues about rendering math on GitHub, but in short:

adavidzh commented 7 years ago

"[…] rendering math inline in blobs happens elsewhere." Elsewhere, where? I.e., where can we go to in order to contribute to fulfilling this need?

hyiltiz commented 7 years ago

How rendering (La)TeX would be a security issue!? It sounds pretty much like Mozilla dropping XUL support and no providing API for some addon functionality (UI) in the new framework (WebExtentions) to addons that need it (e.g. Tag Groups) for "security reasons". It seems UI and LaTeX etc. could hardly produce security issues, unless they have some very weird bug in their code.

AaronNGray commented 7 years ago

The LaTeX code base is a vast and sprawling ancient mess that goes back to the original literate PASCAL TeX release from Knuth which consists of a PASCAL to C converter, and everything else thats been added to the code base over the years. Its next to impossible to secure properly.

AaronNGray commented 7 years ago

Whats needed is a specification of TeX'es semantics and someone to do a modern implimentation really.

AaronNGray commented 7 years ago

I am working towards doing a seemless as possibly image based version that stores TeX in a repo and renders images as a third party solution.

taiya commented 7 years ago

Plus why not just using MathML just like Wikipedia does? (instead of having a full latex spec) it's hard to believe Wikipedia would chose a way of rendering equations that is not safe.