Experience-Monks / math-as-code

a cheat-sheet for mathematical notation in code form
MIT License
15.04k stars 1.08k forks source link

Consider writing this as Jupyter notebooks instead of JavaScript #4

Open blixt opened 9 years ago

blixt commented 9 years ago

GitHub supports executing and rendering Jupyter notebooks which would be much more maintainable and expressive (especially since you can use numpy to execute all kinds of math operations out of the box).

Another big benefit is that you can have GitHub render math notations for you instead of including lots of images in the repo (+ you get retina support!).

Example on GitHub using Jupyter:

mattdesl commented 9 years ago

Ah, nice idea. Related to #2.

Will explore. :+1:

blixt commented 9 years ago

Yup, but unlike Unicode which only lets you sort of reproduce some math notation, you can use LaTeX with Jupyter which is going to be much more recognizable when comparing to papers that also commonly use LaTeX.

It also still supports Markdown so most of what you've written so far can just be transferred over. Finally I would say that it's probably easier to follow maths in Python compared to JavaScript.

tiborsaas commented 9 years ago

I vote for keeping JS. It's very easy to comprehend and the bar to entry is very low.

I didn't know Jupyter before, but looking at that landing page I still don't see it's purpose. For me I would lose interest in this cool project.

blixt commented 9 years ago

@kowdermeister I have a few points to make about your statement, but first look at an example of what this could look like using Jupyter:

https://github.com/blixt/math-as-code/blob/master/Jupyter.ipynb

I think there are clear benefits, the most important one being that the code is being evaluated on the page so you can guarantee that the output matches (which would have prevented a few bug fixes that have already been made). Another benefit is that you get accurate and easily changeable formatting of math notation, both inline and in blocks, that does not require external images.

Anyway, back to my points.

  1. There will be a varied group of people visiting this page, not just JavaScript developers, so I don't think favoring one language or another is a priority. That said, Python syntax is based on how we write pseudo-code and so is probably more natural to read than JavaScript for someone who has no experience in either.
  2. Understanding Jupyter is besides the point as a consumer of this guide. You shouldn't need to know about it if you're just reading the guide. But if you do want to make a contribution, Jupyter takes 5 minutes to get into, because it's essentially a WYSIWYG editor where you write either Markdown (same as the README today) or Python. Try it: https://try.jupyter.org/ (choose New > Python 3)
  3. Why would you lose interest? I think that's an overstatement. The output will be improved in quality and easier to update. If you're just reading the guide you would barely notice a difference – this suggestion is mostly for Matt to have an easier time.
mattdesl commented 9 years ago

Having things run in-line would indeed have solved some of the bugs, but I am going to keep it as JavaScript for a few reasons:

  1. I am most familiar with JS
  2. It is really easy to understand and translate JS (and its limited feature set) to any other (more expressive) language, but not always the other way around.
  3. In addition to the main guide, I hope to provide interactive (HTML5) references eventually, like a WebGL or Canvas demo showing how insights from a paper can be translated into code.
  4. Most importantly, I want to provide succinct and independent modules and/or functions like magnitude that can be used as an implementation reference, and also as a direct dependency for JS developers. Most other languages do not have such highly modular package managers.

I definitely want to solve the image quality, and also would like to automate them via LaTeX somehow.

tiborsaas commented 9 years ago

There's no explanation on Jupyter's site what it is or what it is good for. (I seen it for the first time and totally misunderstood it). Thanks for your explanation now I get it.

https://github.com/blixt/math-as-code/blob/master/Jupyter.ipynb is pretty neat, much less confusing than the long tails one. I'm not familiar with python on this level, but sure, I can look up the language reference.

JS in my opinion is better, because it resembles more to a C style language which I believe is more popular in total.

I also think a guide like this is targeting beginners who are more likely to be familiar with JS thanks to the low entry bar.

This project was one of my wishes in understanding wikipedia pages. I asked it on Quora if there's a site which explains math notation and I was calmed down that it's not possible, because math concepts are deep :)

blixt commented 9 years ago

@mattdesl I understand. Maybe there is something we can do to improve the guide for JavaScript, for example by creating a script that evaluates README.md and makes http://jam3.github.io/math-as-code/ an interactive version.

I also wonder if for LaTeX there is a service that renders a LaTeX string into a math image, which could be used instead of hosting the images on GitHub?

yoshuawuyts commented 9 years ago

I think mathjax and pandoc should get you there. Mathjax uses LaTeX math notation and can be rendered to svg (say, for GitHub), and Pandoc can output its markdown (+mathjax) contents to HTML (for an interactive page). I think they would work pretty well in this case, though some tooling for the build probably needs to be written.

Example

# Section 1

Equation 1:
$$I = \int \rho R^{2} dV$$

See Also

blixt commented 9 years ago

For the fun of it, I created http://tex.la/ which creates SVGs on the fly:

Don't count on it being super scalable right now, but I might make sure that it is if people think it's useful.

mattdesl commented 9 years ago

Great stuff. I'll use that for the time being instead of this one, and will start moving things over to SVG.

caspervonb commented 9 years ago

I doubt GitHub supports it but theoretically jupyter notebooks can be written in JavaScript, since the rename from ipython to jupyter it's a more general focus on getting any language support the kernel and REPL protocol, see https://www.npmjs.com/package/ijs

mattdesl commented 9 years ago

Hmm, sadly it is rendering super small as SVG.

screen shot 2015-06-30 at 7 05 37 pm

Relevant LaTeX:

    f(x)= 
\begin{cases}
    \frac{x^2-x}{x},& \text{if } x\geq 1\\
    0, & \text{otherwise}
\end{cases}

EDIT: Actually, it seems like the editor has a SVG select box I hadn't noticed earlier. It is producing nice results. http://www.codecogs.com/latex/eqneditor.php

blixt commented 9 years ago

Ah yes, you need to set a height or width (see the example at http://tex.la/). I'm going to try to set a proper default scale for the output at some point, but you can also just put an <img> tag with a height attribute:

<img height="50" src="http://tex.la/f(x)%3D%5Cbegin%7Bcases%7D%5Cfrac%7Bx%5E2-x%7D%7Bx%7D%2C%26%20%5Ctext%7Bif%20%7D%20x%5Cgeq%201%5C%5C0%2C%20%26%20%5Ctext%7Botherwise%7D%20%5Cend%7Bcases%7D">

gives:

blixt commented 9 years ago

I fixed the height issue. I don't think it's perfect, but you can now use Markdown image syntax:

![](http://tex.la/A%20%3D%20%5Cbegin%7Bvmatrix%7Da%20%26%20b%20%26%20c%20%26%20d%5C%5Ce%20%26%20f%20%26%20g%20%26%20h%5C%5Ci%20%26%20j%20%26%20k%20%26%20l%5C%5Cm%20%26%20n%20%26%20o%20%26%20p%5Cend%7Bvmatrix%7D)
StoneCypher commented 9 years ago

If the purpose of this guide is to bridge the gap between developers and mathematicians via familiarity, it should be done in a programming language the rest of us are already familiar with, which is not designed to imitate the math that we're trying to disambiguate by pairing it up with traditional code.

I've got ~40 programming languages on my github right now, and I've never even heard of Jupyter.

It may make a sensible second case, but it should not replace something that has a legitimate argument as possibly programming's current lingua franca.

blixt commented 9 years ago

@StoneCypher: I think the argument here is no longer about whether to use Jupyter (which is just a good editor supported by GitHub for Markdown with LaTeX support + interactive shell for Python, by the way – and I'm sure you've worked with Python).

I think we should just discuss how to improve the process of updating the guide, and its appearance (for example: LaTeX in the guide vs. embedded images, and auto-evaluated code to avoid duplication of effort and bugs).

StoneCypher commented 9 years ago

I don't generally use python actually. I spoke other languages in that space before I was aware of it, so there has been no reason for me to take it on, and meaningful whitespace weirds me out.

However, now that you've pointed it out, you're right, that does deflate my commonality comment. I am an outlier there.

mattdesl commented 9 years ago

@blixt it looks like your server is down? Or something?

For now I'm going to stick with linking directly to SVG URLs from this site since it seems very widely used: http://www.codecogs.com/latex/eqneditor.php

Ideally I would rather commit the SVGs so that the images can never break, but there seems to be a bug/limitation in GitHub Markdown that doesn't render relative SVGs. See here and the broken branch here.

I'm going to include the raw LaTeX for each image (currently in Markdown comments). Perhaps later we can build tooling to improve this workflow.

blixt commented 9 years ago

@mattdesl Yeah I made a typo in the last deploy, my bad. :)

Sounds fair, but I would still recommend hotlinking to that service over hosting it in the repo, because GitHub will rehost the images for you on their CDN anyway. This means you can easily update the image from your Markdown, instead of having the additional burden of adding/removing/renaming files.

mattdesl commented 9 years ago

It seems like some SVGs are getting cut off with:

3 \cdot 4 = 3 \times 4

Here's the codecogs result: (top cut off) img

Here's your tex.la result: (bottom cut off) img

Ideally I'd like to hotlink directly to whichever service is (a) most reliable and (b) looks the best. :smile: If either service is going to have frequent downtime then I would rather just deal with the hassle of hosting SVGs in the repo somehow, to ensure all readers have no trouble seeing the content.

blixt commented 9 years ago

Hm yup, I did notice some cut-off as well. I think the values in the viewBox attribute need to be rounded to render properly as images.

The tex.la site is on Google App Engine which means it's very scalable and has great SLAs on uptime, but I wouldn't recommend relying on it right now because the code is just 24 hours old. :) I think the main goal should be to just get rid of manual work around updating the LaTeX (less work = more contributions), so adding in a pre-commit hook that replaces $...$ with generated SVG (either in the repo, or as hotlinks to a service) could be a good thing to do. I might look into it if I get time over, but I'm really swamped these days. :)

daveloyall commented 9 years ago

@blixt I'd like to report a bug on https://github.com/blixt/math-as-code/blob/master/Jupyter.ipynb

rendering is broken, very small letters

That's what it looks like on Firefox 42 on Windows 7.

blixt commented 9 years ago

@daveloyall Huh, that's funky. May want to report that bug to GitHub, along with your browser. I'm not sure where those bugs go, but here's the announcement post: https://github.com/blog/1995-github-jupyter-notebooks-3

Maybe @sshirokov can help out (not sure if mentioning here will summon him)?

sshirokov commented 9 years ago

That's what it looks like on Firefox 42 on Windows 7.

It's a known Firefox-only issue we're working to resolve :crying_cat_face: Sorry about that.

Maybe @sshirokov can help out (not sure if mentioning here will summon him)?

It will :stuck_out_tongue_winking_eye:

@caspervonb: I doubt GitHub supports it but theoretically jupyter notebooks can be written in JavaScript,

It should render just fine regardless of the input language, we're not doing the evaluation, since it's already baked into the notebook, just the rendering.

caspervonb commented 9 years ago

@sshirokov Just noticed that it's all spelled out in JSON, that's awesome.

Correction to the above, ijs seems to have been superseded by ijavascript. Works great.

rreusser commented 9 years ago

Oh man. I wish I'd known tex.la existed before I'd written this: https://github.com/rreusser/gulp-markdown-equations

It's just a build script, so you can do anything you want with it, including swapping it out for a more intelligent parser or rendering to SVG. It's worked great so far.

Well, For the most part. (See: npm/newww#1107 )

Actually I was originally basically gonna create tex.la (which looks great, btw), but didn't love the idea of being on the hook for hosting a domain and running a server for the rest of my life in order to ensure no broken links.

The issue now is threading the needle between the github.com and npmjs.com markdown html sanitizers. (current ideal solution: use SVG, then proxy to cdn.rawgit.com since the goofy security restrictions you've already referenced prevent relative SVG paths, but then we're back to depending upon someone's random external service.)

rreusser commented 9 years ago

Srsly though, why do I always find this stuff after writing it.

StoneCypher commented 9 years ago

@rreusser actually the thing you made is pretty awesome

rreusser commented 9 years ago

@StoneCypher Thanks! It has some issues, but it's pretty modular, so shouldn't be the worst to either fix or substitute the buggy parts. Used it in places like this and have had good luck so far. Way easier than LaTeXiT at any rate!

westurner commented 6 years ago

Given an installed miniconda or anaconda, this should install a Jupyter Notebook, Node, NPM, and the IJavascript Jupyter kernel:

conda install nodejs notebook
npm install -g ijavascript
ijsinstall

Jupyter Notebook supports LaTeX (with MathJax) in markdown:

This is a sigma $\sigma$

If you create a /Dockerfile and a Jupyter notebook, you can launch a cloud instance with https://mybinder.org/

FROM jupyter/base-notebook
RUN conda install -y nodejs notebook
RUN npm install -g ijavascript
RUN ijsinstall

https://github.com/jupyter/docker-stacks/blob/master/base-notebook/Dockerfile

...

LyX is great for editing LaTeX, but not so great for syntax highlighting executable code samples:

https://en.wikipedia.org/wiki/LyX

westurner commented 6 years ago

BinderHub ( https://mybinder.org ) builds containers with repo2docker; which works with a number of different types of config files, including Dockerfile: http://repo2docker.readthedocs.io/en/latest/usage.html#preparing-your-repository