msg-byu / symlib

Spacegroup finder. Includes symmetry-related routines for cluster expansion and other codes that rely on symmetries of lattices and crystals.
MIT License
12 stars 15 forks source link

Incorrect behavior for equal function when comparing rank 1+ arrays #19

Closed wsmorgan closed 5 years ago

wsmorgan commented 6 years ago

Right now if we compare two arrays we get the following, in the following example they are a and b:

 a   4.5600927478517406E-009  -2.9999999934510697        3.9999999999953424     
 b           0          -3           4
 atol   1.0000000000000000E-010
 rtol   1.0000000000000000E-003
 rtol * max(abs(a),abs(b))  = (4.5600927478517410E-012   3.0000000000000001E-003   4.0000000000000001E-003)
 atol + rtol * max(abs(a),abs(b)) =   (1.0456009274785174E-010   3.0000001000000001E-003   4.0000000999999997E-003)
 abs(a-b) = (4.5600927478517406E-009   6.5489302869536914E-009   4.6576076329074567E-012)
 abs(a-b) < atol + rtol * max(abs(a),abs(b)) =  F T T

and because there is an F in the results the algorithm returns False, however, from our own observations we can clearly see that the result should be True. The problem seems to be that we are using max an arrays which results in another array that is then compared to. The solution is to use maxval on each array then wrap the results in max like so:

atol + rtol * max(maxval(abs(a)), maxval(abs(b))) =  4.0000000999999997E-003
abs(a-b) < atol + rtol * max(maxval(abs(a)), maxval(abs(b))) = T T T