minor test failure in test_psm.py

[ChrisBarker-NOAA]

I got one failed test:

    # Compare results to the correct answer

  assert de_max_gas_model == de_max_gas

E assert 0.033172284658376426 == 0.03294393791256645 E +0.033172284658376426 E -0.03294393791256645

../../../../ERD-PythonPackages/tamoc/tamoc/test/test_psm.py:200: AssertionError

looks like a significant figures issue with maybe a different compiler or library -- it matches to less than 1% error.

Though with double precision, that's quite a bit -- it's even a fair bit with single precision, so I'm not sure if I should ignore it.

OS-X 10.14
Python3.8
In [2]: numpy.__version__
Out[2]: '1.20.2'
In [5]: scipy.__version__
Out[5]: '1.6.2'

GNU Fortran (GCC) 9.3.0

Apple clang version 11.0.0 (clang-1100.0.33.17)

[socolofs]

I’ll look into this one. This part of the code is solving a non-linear root-finding problem. I don’t really know why, but it does seem that scipy iteration methods converge on results that differ in the 2nd or 3rd significant figure on different machines whenever iteration is involved. Even though the convergence criteria of the iteration is probably 1e-6 or 1e-8, it doesn’t change the fact that these converge on different answers. It’s been a frustration of unit testing throughout TAMOC. I confirmed this morning that this test is passing on my Mac Pro.

Scott

---

Scott A. Socolofsky, A.P. and Florence Wiley Professor II, Zachry Department of Civil and Environmental Engineering Texas A&M University (979) 845-4517

http://ceprofs.civil.tamu.edu/ssocolofsky

On Apr 2, 2021, at 8:38 PM, Chris Barker @.**@.>> wrote:

I got one failed test:

# Compare results to the correct answer

assert de_max_gas_model == de_max_gas

E assert 0.033172284658376426 == 0.03294393791256645 E +0.033172284658376426 E -0.03294393791256645

../../../../ERD-PythonPackages/tamoc/tamoc/test/test_psm.py:200: AssertionError

looks like a significant figures issue with maybe a different compiler or library -- it matches to less than 1% error.

Though with double precision, that's quite a bit -- it's even a fair bit with single precision, so I'm not sure if I should ignore it.

OS-X 10.14 Python3.8 In [2]: numpy.version Out[2]: '1.20.2' In [5]: scipy.version Out[5]: '1.6.2'

GNU Fortran (GCC) 9.3.0

Apple clang version 11.0.0 (clang-1100.0.33.17)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/socolofs/tamoc/issues/11__;!!KwNVnqRv!QPwNsmKXitjgx1WAKbIAx4TT_bOjaMDnXwwEjk2u197sz4LWL0kPqFTXsbwY2B7L$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AA6XESFXH3JPLRVIX27ZVHTTGZWS3ANCNFSM42JV5UYA__;!!KwNVnqRv!QPwNsmKXitjgx1WAKbIAx4TT_bOjaMDnXwwEjk2u197sz4LWL0kPqFTXsew8UoR2$.

[ChrisBarker-NOAA]

Is it using double or single precision?

For single, I suppose three digits isn't bad.

but yes, very odd that a convergence criteria of 1e-8 would result in differences that large.

I'm also very confused about the difference on different machines -- every machine we commonly use used IEEE 754 FP arithmetic -- the results should be exactly the same.

Maybe different compiler optimizations ?!?

It is a real challenge to test this sort of thing, but I generally figure that a real error would result in differences larger than even 1% or so, so testing to a tolerance of 1e-3 or so should catch the stuff we really want to catch.