search

fredrik-johansson / fungrim

Fungrim: the Mathematical Functions Grimoire
MIT License
119 stars 15 forks source link

Scope of Fungrim #9

Open certik opened 5 years ago

certik commented 5 years ago

Can Fungrim be used to store any formulas, or do you intend to limit its scope? It seems there is an infinite number of valid formulas...

As an example, I have this page:

https://www.theoretical-physics.com/dev/math/complex.html

where I had to painfully derive (and check with Python code!) many formulas that are valid in the whole complex plane, using the Wolfram functions site and my own derivations. Would it make sense to port such formulas into Fungrim, and then I can have Python scripts to retrieve them and put them into my page? That way there can be a reference/link of such formulas into Fungrim for more information, and symbolic code to be used with computer algebra systems.

fredrik-johansson commented 5 years ago

That's a great question.

It depends on what (if anything) people start using it for, but as a general principle, I would say that formulas are interesting if they are useful in real computations (pen and paper, symbolic or numerical). Most of the formulas in the mathematical part of your theoretical physics reference are certainly in scope. The current ID scheme allows up to 16.7 million entries, so that's one (temporary) limit :)

It's perhaps more a question of how you organize the formulas. A good reference work doesn't just throw formulas in lexicographic order at the reader. There should be some structure, with more important formulas first, and less important formulas relegated to sub-topics (and sub-sub-topics). If you grow the collection of formulas more breadth/importance-first than depth-first, I think this works out naturally.

Any useful formula can of course be turned into endless variants that essentially say the same thing. I think what makes sense to include depends on importance, which is a bit of a judgement call. For example, I manipulate expressions with the gamma function so often that I'd be happy to see a listing of all the versions of the recurrence relation with different combinations of z+1, z, and z-1 on the left and right hand sides; it's convenient if I can look up the right version directly without having to manipulate it in my head (I'm prone to making off-by-one or sign errors). But for a less common function, probably writing down a single version of a recurrence relation is good enough.

certik commented 5 years ago

Ok, so I just added one sample formula from my page in https://github.com/fredrik-johansson/fungrim/pull/11.

I think it should be possible to improve SymPy to be able to manipulate formulas from Fungrim, correctly. That would be super helpful.

certik commented 5 years ago

People have been asking me to make the formulas on my theoretical physics page accessible via sympy. I didn't quite know how to approach it, but I finally figured it out, at least for the complex.html page. Fungrim can store the essential formulas, and then either Fungrim or SymPy will have a module that allows substitution of one Fungrim formula into another (with proper handling of assumptions), which will allow correct manipulation. Then the derivations, say, at https://www.theoretical-physics.com/dev/math/complex.html#exponential can be done by simply referencing hashes in Fungrim and substitution. That should be 100% robust and correct.

And then it's just about how to visualize it. For example it can look like it does now, but there can be links to Fungrim for individual formulas, as well as there can be a link to step by step instructions how exactly the substitution was done. And there can be link to a SymPy expression that one can then readily manipulate (this could perhaps by done via Fungrim, which can have SymPy, Mathematica, Maple, Sage, ... code for each formula).

What do you think?

certik commented 5 years ago

P.S. According to http://functions.wolfram.com/, they have 300K formulas and in my experience they cover almost everything I ever needed. However, they typically don't have exactly the formula that I would want on my website, but they have an equivalent formula. So let's say each formula has 10 useful ways of writing it, that's 3M. So it's seems it could be practical to include all formulas that people might need in books, webpages, articles, etc. Together with automatic formula rewriting/substitutions described in my previous post above, I think this might cover everything.

The only issue is that if there are going to be millions of formulas, they probably can't be stored in Python source files like now, but rather in some kind of a database in a semantically equivalent way.

fredrik-johansson commented 5 years ago

Note that about 90% of the Wolfram formulas are automatically-generated entries of the following type:

If you have algorithms that can do the evaluations, it's perhaps not necessary to list that many special cases for manual lookup. In particular, hypergeometric formulas are readily obtained by applying recurrence relations to a small number of special cases. It remains to figure out what is actually useful. I would like to start by scaling Fungrim up to 10000 mostly non-redundant formulas before thinking about what to do with 100K or 1M formulas :)