search

pbrod / nvector

Nvector is a suite of tools written in Python to solve geographical position calculations.
Other
57 stars 7 forks source link

use of np.isclose to compare number close to 0 #12

Closed nicolas-UKLSL closed 3 years ago

nicolas-UKLSL commented 3 years ago

According to numpy documentation (v1.19), isclose should not be used with default atol value for comparing numbers close to 0. See https://numpy.org/doc/stable/reference/generated/numpy.isclose.html

However, this is what is done in function on_great_circle in _core.py (and maybe other areas).

pbrod commented 3 years ago

Well it depends on the scale of the application and the desired accurracy. In nvector the scale is on the order of the radius of the earth which is about 6.3710^6 meters. In the on_great_circle function np.isclose checks if absolute(distance - 0) <= (atol + rtol absolute(0)) = atol = 1e-8 In other words if the position differ with 10^-8 meters it is on the great circle path. I would say that is reasonable. Actually it is calculated to machine precision because atol is approximately equal to the earth radius times 10^-14.
For my applications I would even accept it to be on the great circle if the position differ with less than 1 meter, which corresponds to a single precison calculation.

nicolas-UKLSL commented 3 years ago

You are right, though because 0 is the second argument, rtol is not used. I would think it best to change from np.isclose(distance, 0) to np.isclose(0, distance)

pbrod commented 3 years ago

The rtol parameter does not matter at all when you compare to zero: absolute(0 - distance) <= (atol + rtol absolute(distance)) = 1e-8 + 1e-6 distance => distance <= atol/(1-rtol) = 1e-8/(1-1e-6) = 1.000001000001e-08

pbrod commented 3 years ago

Hmmm... I think I will remove the rtol parameter since it is useless in this situation, but I am wondering if the default atol value is too strict. For practical application of gps data the error ranges from 0.01 meter to 15 meters according to https://gis.stackexchange.com/questions/43617/what-is-the-maximum-theoretical-accuracy-of-gps.

pbrod commented 3 years ago

The documentation of np.isclose states that the default value of atol=1e-8 is appropriate for expected values of order unity and I think that is the major use case for this library.

nicolas-UKLSL commented 3 years ago

Yes, I think this is fair

From: Per A. Brodtkorb notifications@github.com Sent: 11 December 2020 16:10 To: pbrod/nvector nvector@noreply.github.com Cc: Nicolas Croisard nicolas@uklsl.com; Author author@noreply.github.com Subject: Re: [pbrod/nvector] use of np.isclose to compare number close to 0 (#12)

The documentation of np.isclose states that the default value of atol=1e-8 is appropriate for expected values of order unity and I think that is the major use case for this library.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/pbrod/nvector/issues/12#issuecomment-743282778, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARW3HRYAUSX3KVI5RAX3ZCLSUI77FANCNFSM4UVD72HQ.