numworks / epsilon

Modern graphing calculator operating system.
https://www.numworks.com/resources/engineering/software/
1.75k stars 463 forks source link

tan() has degraded precision in the Calculation app. #2043

Open goualard-f opened 1 year ago

goualard-f commented 1 year ago

Describe the bug

In the "Calculation" app, the tan() function returns undef for some arguments that are computable. Maybe a duplicate of issue #1386?

Screenshots

screenshot(2)

To Reproduce

Steps to reproduce the behavior:

  1. Go to the 'Calculation' app
  2. Type 'tan(642336890023956)'
  3. See error

Expected behavior

With the Python app. of the calculator, we get:

>>> from math import tan
>>> tan(642336890023956)
1285230487114249.8

The value 642336890023956 is very close to a multiple of $\pi/2$:

$642336890023956\approx \pi\times204462182355175.4999999999999997523324\cdots$

which must be the source of the problem. However, that does not explain the degraded precision with respect to what is computed with other mathematical libraries.

Environment

pitonyak commented 1 year ago

I have verified the behavior.

First, I tested using LibreOffice Calc and I received a similar (but different) value (1285230487114206). Then I tried on my HP Prime in both Home and CAS mode, which returned different negative numbers (like -74 and -54 or similar), probably based on the HP precision.

I do see that NumWorks should be using double precision (see https://www.numworks.com/calculator/update/version-01).

I then tested to receive undef from the standard calculator and from Python I now see 1285230487114250, so 0.2 greater than in the example above.

I am curious if the problem is related to converting the input or in the calculation library.