exp(x) doesn't equal e^x

[panuhorsmalahti]

math.exp(10)
 22026.465794806718

math.eval('e^10')
 22026.465794806707

Native result:

Math.exp(10)
 22026.465794806718

There's a small rounding issue somewhere.

[josdejong]

This is an inescapable issue that comes with working with floating point numbers: you get round-off errors. JavaScript Numbers have a precision of almost 16 decimals.

If you don't want this, there are a few options:

• Don't use Number but use BigNumber for arbitrary precision calculations. See docs on BigNumbers: http://mathjs.org/docs/datatypes/bignumbers.html
• The comparison functions of math.js consider numbers with a very small round-off error as equal, so when math.eval('exp(10) == e^10') will return true. To prevent round-off errors to show up in a user interface, you can round the output to say 14 digits, like math.format(0.1 + 0.2, {precision: 14}) will output 0.3 instead of 0.30000000000000004. See docs on round-off errors and equality: http://mathjs.org/docs/datatypes/numbers.html

[panuhorsmalahti]

Hmm, the same problem is with native .exp and .pow, e.g.

Math.pow(Math.E, 10) === Math.exp(10)
 false

I don't quite see why they differ. I read that in C# they're equal.

[josdejong]

They are mathematically equal, but the numerical implementation can differ. There are may different algorithms to calculate pow or exp. You could implement exp(x) as pow(e, x), but I can imagine there are faster algorithms to calculate this specific case of exponentiation.

Anyway, apparently they are not the same in JavaScript. Why is this a problem?

[panuhorsmalahti]

Hmm, I just tried this in C#, and they're not equal there either. It seems that the numerical implementations differ in .NET world too.

Interestingly the commentators here claim that "they're equal": http://stackoverflow.com/questions/18367240/math-pow-vs-math-exp-c-sharp-net

Even Eric Lippert says the "identity is true"..

[ComFreek]

@panuhorsmalahti The identity is true. If you refer to this comment, the equation given is to be interpreted in a more mathematical sense. The comment in the corresponding answer of the linked comment above also mentions in the provided code that "Math.Pow(Math.E,n) = Math.Exp(n)" is just a "human equation".

As @josdejong said, it's just a matter of computation / of the different algorithms which get internally used.

[panuhorsmalahti]

@ComFreek Yeah, "obviously" it's true in the mathematical sense. Unfortunately JavaScript (and apparently C# too) language specifies that many mathematical functions are implementation specific and are defined to only be approximations. C# is a bit more complicated (=worse), because the standard doesn't even specify the precision for floating point numbers.

http://es5.github.io/#x15.8.2

The standard states that many functions are approximations. The crucial point is, however, that this doesn't mean that math.js should necessarily follow the JavaScript standard. It might be a lot of trouble, but math.js could actually be specified and implemented more precisesly so that exp(10) === e^10. The current math.js implementation does seem to use the Math functions in many places, but the algorithms could be implemented from scratch.

EDIT: The implementation for Firefox is here: https://hg.mozilla.org/mozilla-central/file/cadcd47db610/js/src/jsmath.cpp

It seems that the Math.pow implementation has a optimization for the case when y is an integer (which is the test case here). That's not the root cause, though, as Math.exp(9.99) doesn't equal Math.pow(Math.E, 9.99). Anyway, the code appears to use different implementations for these functions.

[ComFreek]

@panuhorsmalahti Numerical approximations are just easier to handle than precise mathematical expressions. I think the big problem you will eventually be hitting with 100% accurate expressions is the comparison of them. You have to bring them to a common denominator.

Algorithms written in JS are probably a lot slower than their built-in C counterparts in the underlying VMs. In addition, where would you draw the line between approximation and precision? One will always hit corner cases for a specific n where exp(n) !== e^n. Maybe one could transform e^n to exp(n) during parsing. However, this might lead to unexpected results (e.g. when one is comparing a series of e^n calls with n not only being a whole number).

[josdejong]

It might be a lot of trouble, but math.js could actually be specified and implemented more precisesly so that exp(10) === e^10

@panuhorsmalahti please read my first comment again, math.js already contains what you ask for: Math.js has BigNumber support, which means you can execute exp and pow with arbitrary precision instead of using the build in Math.exp and Math.pow using 64 bit floating point Numbers.

You will see that if you evaluate both exp(10) and e^10 using BigNumbers, they still have a round-off error. This is because the exact results of these expressions have an infinite number of decimals. So when you represent this result with either a Number (~16 decimals) or BigNumber (64 decimals by default, configurable), you always have an inexact approximation. You would have to set the number of decimals to infinity, calculate the results, and compare that to be entirely sure of equality. However, that's not really feasible.

I hope you understand the limitations of numerical calculations (both with Numbers and with BigNumbers). I think what you are looking for is to solve these equations algebraically instead of numerically, you want a Computer Algebra System. Math.js doesn't have support for that, but it's on the wish list, see https://github.com/josdejong/mathjs/issues/87 and related topics.

What is you use case exactly?

[panuhorsmalahti]

This is because the exact results of these expressions have an infinite number of decimals.

That's not the root cause. You could actually properly compute the value of exp(10) and e^10 to the same exact binary representation, and they would equal. If you use the right algorithms, both expression would return the same value. However, the same algorithms are not used here (the main reason is probably performance).

Apparently the value is approximately 22026.46579480671651696 according to http://keisan.casio.com/calculator.

This is rounded to 22026.465794806718 in Javascript. Therefore (assuming that online calculator can be trusted), Math.exp returns the correct result, but Math.pow is slightly wrong. Of course, the standard allows Math.pow to be an approximation, but 22026.465794806718 is still the correct result.

Unfortunately GNU Octave's 15 decimals is not precise enough to verify this.

The fact that e^10 has infinite number of decimals is not really the cause here. The cause is that Math.pow doesn't use a precise enough algorithm.

[josdejong]

I suggest your read my previous comments and intensively study how floating point numbers and floating point arithmetic works before commenting further on this issue.

JavaScript numbers are regular double floats (64 bit), as do most programming languages and calculators. Double floats have a precision of almost 16 decimals. So the first 15 decimals are reliable, everything behind that is not. In your case the first 15 decimals are 22026.4657948067, and the rest, 18 is unreliable. When you only look at the reliable number of decimals of a floating point, both results are correct and equal. Now you may guess why GNU Octave outputs 15 decimals at most...

Third time: what are you trying to achieve? Equality with floating point numbers is per definition a tricky thing (try 0.1+0.2==0.3 in your JavaScript console). Why do you try to achieve this equality numerically rather than algebraically?

[panuhorsmalahti]

Third time: what are you trying to achieve?

Well, I'd like to use a math library which evaluates to the highest precision possible. It is certainly possible to reliably have the best precision possible. I showed one example of how to do it previously: first compute the result using a higher precision, then round to JavaScript's precision.

Brendan replied to this issue on twitter: https://twitter.com/BrendanEich/status/529794685342994433

Perhaps I should post on es-discuss about this.

[josdejong]

And why do you want to evaluate "with the highest precision possible"? Are you only interested in this specific exp(x) !== e^x problem, or are you looking at numerical accuracy in a broader sense?

The reason I sometimes want to work with a higher accuracy is do that is to get rid of simple round-off errors. I haven't come across any practical cases where I actually need say pi or exp(x) in hundreds or thousands of digits accurately, the regular 15 digits is normally fine.

I agree it would be nicer when exp(x) !== e^x would return true. If there are possibilities to improve that in a next version of JavaScript that would be great. However, it's not just this issue: you have the same with 0.1+0.2 !== 0.3, sin(pi) !== 0, etc, etc. There are plenty of cases where you have accuracy issues, caused by performing operations on numbers with limited accuracy. Note that Douglas Crockford proposed an alternative for the floating point number to solve a lot of these round-off errors: DEC64, can be interesting.

I'd like to use a math library which evaluates to the highest precision possible.

With math.js you can: use BigNumbers. You can choose the precision as you please. Did you give this a try already?

(I don't think you really mean the highest precision possible though, that would mean representing numbers with an infinite number of digits in there entirety, which would require quite some memory you know ;) )

first compute the result using a higher precision, then round to JavaScript's precision.

Exactly. If you round the output to a few digits less than the accuracy of the number type that you use, you can "hide" these round-off errors for the users eye. But it does not really solve the issue, equality checks would fail for example.

If you want calculations without accuracy issues you can't use regular Numbers. You can use BigNumbers with a huge number digits of to solve some of the inaccuracy issues, but this does not fundamentally solve your precision issue: numbers with an infinite number of digits never fit in a floating point number with a limited number of digits, and thus you introduce round-off errors when operation on them. The only way you can solve this issue fundamentally is by doing algebraic calculations instead of numerical.

Can you explain more on why you want to do numeric calculations with extra high accuracy? Shouldn't you look into CAS (computer algebra systems)?

[josdejong]

Maybe an easier way to explain it: Suppose exp and pow would use the same algorithm, so the issue exp(10) !== e^10 would have bin solved. Can you then solve this next issue for me?

exp(4) * exp(6) !== exp(10)

@panuhorsmalahti I hope you have a little bit better idea on the issues with floating point numbers. Can we close this issue?

[panuhorsmalahti]

I hope you have a little bit better idea on the issues with floating point numbers.

Not really, I already knew this stuff.

exp(4) * exp(6) !== exp(10)

Solving that is another issue.

[josdejong]

Solving that is another issue.

Just think about it. It may be a relevant and surprisingly similar issue in your quest for "a math library which evaluates with the highest precision possible".

Not really, I already knew this stuff.

Ok then, I will stop wasting my time here. I just try to help you, but all you do is ignoring the questions I ask trying to get your use case clear and explaining the context.

Have a nice day.