OMS-NetZero / FAIR

Finite-amplitude Impulse Response simple climate model
https://docs.fairmodel.net
Apache License 2.0
123 stars 62 forks source link

Fix tests for array equality using Numpy helper #22

Closed rgieseke closed 6 years ago

rgieseke commented 6 years ago

I'm getting occasional test failures for

tests/test_fair.py::test_division

and

tests/test_fair.py::test_co2_scale

due to float precision problems, so differences like

print(C2[:,0]  - C1[:,0])

look like

0.00000000e+00  0.00000000e+00  0.00000000e+00  5.68434189e-14
 0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
 0.00000000e+00 -5.68434189e-14  0.00000000e+00  0.00000000e+00

This pull request is incomplete, as it doesn't cover yet the "greater than" comparisions -- not sure how to handle that, round before comparing or use both assert_allclose and assert_array_less somehow?

rgieseke commented 6 years ago

Sorry, this also removes some trailing whitespace, see the whitespace ignoring diff here:

https://github.com/rgieseke/FAIR/commit/4ef5cb717614eaadbd2f395e4846facd54f5703f?w=1

rgieseke commented 6 years ago

It seems that dropping the pickle dependency in

https://github.com/OMS-NetZero/FAIR/commit/67695b6bbf21822852ea012c06c806b66555b02d

also fixed this testing problem. I only got this error on Mac, not on Linux, so I assume reading the pickle file from a different platform caused this.

For future reference, I used this script for testing:

#!/bin/bash
MAX_TRIES=5
ERRORED=0

for (( i=1; i<=$MAX_TRIES; i++ ))
  do
    pytest tests/test_fair.py::test_division --verbose
    if [ $? -eq 1 ]; then
      ERRORED=$((ERRORED + 1))
    fi
  done
echo $ERRORED of $MAX_TRIES failed.
if [ $ERRORED -eq 0 ]; then
  exit 0
fi
exit 1