asdf-format / asdf

ASDF (Advanced Scientific Data Format) is a next generation interchange format for scientific data
http://asdf.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
523 stars 57 forks source link

`assert_tree_match` and `np.testing` ignores array masks #1806

Open braingram opened 2 months ago

braingram commented 2 months ago

While reviewing https://github.com/asdf-format/asdf/pull/1803 a somewhat related issue was found. See https://github.com/asdf-format/asdf/pull/1803#issuecomment-2236535510 for the initial comment.

asdf._tests._helpers.assert_tree_match discards array masks during the comparison. This is caused by 2 related issues (both covered here) and this line: https://github.com/asdf-format/asdf/blob/33e7095df0c58705043de8f75497dfd64d04bbfb/asdf/_tests/_helpers.py#L55 The call to __array__ drops the mask:

import numpy as np
arr = np.arange(5)
m0 = np.ma.masked_array(arr, False)
a2 = m0.__array__()
print(f"{type(a2)}: {a2}")

returns

<class 'numpy.ndarray'>: [0 1 2 3 4]

So the mask never makes it to assert_array_equal. Furthermore:

import numpy as np
arr = np.arange(5)
m0 = np.ma.masked_array(arr, True)
m1 = np.ma.masked_array(arr, False)
np.testing.assert_array_equal(m0, m11)

Does not fail (as it appears assert_array_equal (and assert_equal) ignore masks.

As our test suite contains several masked arrays it seems worthwhile to: