search

dask / crick

Streaming and approximate algorithms. WIP, use at own risk.
BSD 3-Clause "New" or "Revised" License
21 stars 15 forks source link

0.0.5: pytest fails in few units #45

Closed kloczek closed 7 months ago

kloczek commented 7 months ago

Describe the issue: Look like pytest fails in few units assertions.

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

Here is pytest output: ```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-crick-0.0.5-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-crick-0.0.5-2.fc35.x86_64/usr/lib/python3.8/site-packages + /usr/bin/pytest -ra -m 'not network' --import-mode=importlib ============================= test session starts ============================== platform linux -- Python 3.8.18, pytest-7.4.3, pluggy-1.3.0 rootdir: /home/tkloczko/rpmbuild/BUILD/crick-0.0.5 configfile: setup.cfg collected 97 items crick/tests/test_space_saving.py ............................... [ 31%] crick/tests/test_stats.py .......FF...FF...FF...FF...FF...FF....... [ 74%] crick/tests/test_tdigest.py ......................... [100%] =================================== FAILURES =================================== ______________________________ test_skew[True-x2] ______________________________ x = array([1]), bias = True @pytest.mark.parametrize( "x", [ normal, empty, one, duplicate, different, ], ) @pytest.mark.parametrize("bias", [True, False]) def test_skew(x, bias): stats = pytest.importorskip("scipy.stats") s = SummaryStats() s.update(x) res = s.skew(bias=bias) sol = stats.skew(x[~np.isnan(x)], bias=bias) if len(x) else np.nan > np.testing.assert_allclose(res, sol, rtol=RTOL, atol=ATOL) crick/tests/test_stats.py:63: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = (.compare at 0x7f2a6d079f70>, array(nan), array(0.)) kwds = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=0, atol=1e-08', 'verbose': True} @wraps(func) def inner(*args, **kwds): with self._recreate_cm(): > return func(*args, **kwds) E AssertionError: E Not equal to tolerance rtol=0, atol=1e-08 E E x and y nan location mismatch: E x: array(nan) E y: array(0.) /usr/lib64/python3.8/contextlib.py:75: AssertionError ______________________________ test_skew[True-x3] ______________________________ x = array([1, 1]), bias = True @pytest.mark.parametrize( "x", [ normal, empty, one, duplicate, different, ], ) @pytest.mark.parametrize("bias", [True, False]) def test_skew(x, bias): stats = pytest.importorskip("scipy.stats") s = SummaryStats() s.update(x) res = s.skew(bias=bias) sol = stats.skew(x[~np.isnan(x)], bias=bias) if len(x) else np.nan > np.testing.assert_allclose(res, sol, rtol=RTOL, atol=ATOL) crick/tests/test_stats.py:63: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = (.compare at 0x7f2bf1dfa5e0>, array(nan), array(0.)) kwds = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=0, atol=1e-08', 'verbose': True} @wraps(func) def inner(*args, **kwds): with self._recreate_cm(): > return func(*args, **kwds) E AssertionError: E Not equal to tolerance rtol=0, atol=1e-08 E E x and y nan location mismatch: E x: array(nan) E y: array(0.) /usr/lib64/python3.8/contextlib.py:75: AssertionError _____________________________ test_skew[False-x2] ______________________________ x = array([1]), bias = False @pytest.mark.parametrize( "x", [ normal, empty, one, duplicate, different, ], ) @pytest.mark.parametrize("bias", [True, False]) def test_skew(x, bias): stats = pytest.importorskip("scipy.stats") s = SummaryStats() s.update(x) res = s.skew(bias=bias) sol = stats.skew(x[~np.isnan(x)], bias=bias) if len(x) else np.nan > np.testing.assert_allclose(res, sol, rtol=RTOL, atol=ATOL) crick/tests/test_stats.py:63: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = (.compare at 0x7f2a6a63c310>, array(nan), array(0.)) kwds = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=0, atol=1e-08', 'verbose': True} @wraps(func) def inner(*args, **kwds): with self._recreate_cm(): > return func(*args, **kwds) E AssertionError: E Not equal to tolerance rtol=0, atol=1e-08 E E x and y nan location mismatch: E x: array(nan) E y: array(0.) /usr/lib64/python3.8/contextlib.py:75: AssertionError _____________________________ test_skew[False-x3] ______________________________ x = array([1, 1]), bias = False @pytest.mark.parametrize( "x", [ normal, empty, one, duplicate, different, ], ) @pytest.mark.parametrize("bias", [True, False]) def test_skew(x, bias): stats = pytest.importorskip("scipy.stats") s = SummaryStats() s.update(x) res = s.skew(bias=bias) sol = stats.skew(x[~np.isnan(x)], bias=bias) if len(x) else np.nan > np.testing.assert_allclose(res, sol, rtol=RTOL, atol=ATOL) crick/tests/test_stats.py:63: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = (.compare at 0x7f2bf1dfa670>, array(nan), array(0.)) kwds = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=0, atol=1e-08', 'verbose': True} @wraps(func) def inner(*args, **kwds): with self._recreate_cm(): > return func(*args, **kwds) E AssertionError: E Not equal to tolerance rtol=0, atol=1e-08 E E x and y nan location mismatch: E x: array(nan) E y: array(0.) /usr/lib64/python3.8/contextlib.py:75: AssertionError ___________________________ test_kurt[True-True-x2] ____________________________ x = array([1]), bias = True, fisher = True @pytest.mark.parametrize( "x", [ normal, empty, one, duplicate, different, ], ) @pytest.mark.parametrize("bias", [True, False]) @pytest.mark.parametrize("fisher", [True, False]) def test_kurt(x, bias, fisher): stats = pytest.importorskip("scipy.stats") s = SummaryStats() s.update(x) res = s.kurt(bias=bias, fisher=fisher) if len(x): sol = stats.kurtosis(x[~np.isnan(x)], bias=bias, fisher=fisher) else: sol = np.nan > np.testing.assert_allclose(res, sol, rtol=RTOL, atol=ATOL) crick/tests/test_stats.py:88: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = (.compare at 0x7f2a6d079ee0>, array(nan), array(-3.)) kwds = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=0, atol=1e-08', 'verbose': True} @wraps(func) def inner(*args, **kwds): with self._recreate_cm(): > return func(*args, **kwds) E AssertionError: E Not equal to tolerance rtol=0, atol=1e-08 E E x and y nan location mismatch: E x: array(nan) E y: array(-3.) /usr/lib64/python3.8/contextlib.py:75: AssertionError ___________________________ test_kurt[True-True-x3] ____________________________ x = array([1, 1]), bias = True, fisher = True @pytest.mark.parametrize( "x", [ normal, empty, one, duplicate, different, ], ) @pytest.mark.parametrize("bias", [True, False]) @pytest.mark.parametrize("fisher", [True, False]) def test_kurt(x, bias, fisher): stats = pytest.importorskip("scipy.stats") s = SummaryStats() s.update(x) res = s.kurt(bias=bias, fisher=fisher) if len(x): sol = stats.kurtosis(x[~np.isnan(x)], bias=bias, fisher=fisher) else: sol = np.nan > np.testing.assert_allclose(res, sol, rtol=RTOL, atol=ATOL) crick/tests/test_stats.py:88: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = (.compare at 0x7f2a6d08db80>, array(nan), array(-3.)) kwds = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=0, atol=1e-08', 'verbose': True} @wraps(func) def inner(*args, **kwds): with self._recreate_cm(): > return func(*args, **kwds) E AssertionError: E Not equal to tolerance rtol=0, atol=1e-08 E E x and y nan location mismatch: E x: array(nan) E y: array(-3.) /usr/lib64/python3.8/contextlib.py:75: AssertionError ___________________________ test_kurt[True-False-x2] ___________________________ x = array([1]), bias = False, fisher = True @pytest.mark.parametrize( "x", [ normal, empty, one, duplicate, different, ], ) @pytest.mark.parametrize("bias", [True, False]) @pytest.mark.parametrize("fisher", [True, False]) def test_kurt(x, bias, fisher): stats = pytest.importorskip("scipy.stats") s = SummaryStats() s.update(x) res = s.kurt(bias=bias, fisher=fisher) if len(x): sol = stats.kurtosis(x[~np.isnan(x)], bias=bias, fisher=fisher) else: sol = np.nan > np.testing.assert_allclose(res, sol, rtol=RTOL, atol=ATOL) crick/tests/test_stats.py:88: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = (.compare at 0x7f2a6d08dc10>, array(nan), array(-3.)) kwds = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=0, atol=1e-08', 'verbose': True} @wraps(func) def inner(*args, **kwds): with self._recreate_cm(): > return func(*args, **kwds) E AssertionError: E Not equal to tolerance rtol=0, atol=1e-08 E E x and y nan location mismatch: E x: array(nan) E y: array(-3.) /usr/lib64/python3.8/contextlib.py:75: AssertionError ___________________________ test_kurt[True-False-x3] ___________________________ x = array([1, 1]), bias = False, fisher = True @pytest.mark.parametrize( "x", [ normal, empty, one, duplicate, different, ], ) @pytest.mark.parametrize("bias", [True, False]) @pytest.mark.parametrize("fisher", [True, False]) def test_kurt(x, bias, fisher): stats = pytest.importorskip("scipy.stats") s = SummaryStats() s.update(x) res = s.kurt(bias=bias, fisher=fisher) if len(x): sol = stats.kurtosis(x[~np.isnan(x)], bias=bias, fisher=fisher) else: sol = np.nan > np.testing.assert_allclose(res, sol, rtol=RTOL, atol=ATOL) crick/tests/test_stats.py:88: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = (.compare at 0x7f2a6a511430>, array(nan), array(-3.)) kwds = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=0, atol=1e-08', 'verbose': True} @wraps(func) def inner(*args, **kwds): with self._recreate_cm(): > return func(*args, **kwds) E AssertionError: E Not equal to tolerance rtol=0, atol=1e-08 E E x and y nan location mismatch: E x: array(nan) E y: array(-3.) /usr/lib64/python3.8/contextlib.py:75: AssertionError ___________________________ test_kurt[False-True-x2] ___________________________ x = array([1]), bias = True, fisher = False @pytest.mark.parametrize( "x", [ normal, empty, one, duplicate, different, ], ) @pytest.mark.parametrize("bias", [True, False]) @pytest.mark.parametrize("fisher", [True, False]) def test_kurt(x, bias, fisher): stats = pytest.importorskip("scipy.stats") s = SummaryStats() s.update(x) res = s.kurt(bias=bias, fisher=fisher) if len(x): sol = stats.kurtosis(x[~np.isnan(x)], bias=bias, fisher=fisher) else: sol = np.nan > np.testing.assert_allclose(res, sol, rtol=RTOL, atol=ATOL) crick/tests/test_stats.py:88: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = (.compare at 0x7f2a6a511a60>, array(nan), array(0.)) kwds = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=0, atol=1e-08', 'verbose': True} @wraps(func) def inner(*args, **kwds): with self._recreate_cm(): > return func(*args, **kwds) E AssertionError: E Not equal to tolerance rtol=0, atol=1e-08 E E x and y nan location mismatch: E x: array(nan) E y: array(0.) /usr/lib64/python3.8/contextlib.py:75: AssertionError ___________________________ test_kurt[False-True-x3] ___________________________ x = array([1, 1]), bias = True, fisher = False @pytest.mark.parametrize( "x", [ normal, empty, one, duplicate, different, ], ) @pytest.mark.parametrize("bias", [True, False]) @pytest.mark.parametrize("fisher", [True, False]) def test_kurt(x, bias, fisher): stats = pytest.importorskip("scipy.stats") s = SummaryStats() s.update(x) res = s.kurt(bias=bias, fisher=fisher) if len(x): sol = stats.kurtosis(x[~np.isnan(x)], bias=bias, fisher=fisher) else: sol = np.nan > np.testing.assert_allclose(res, sol, rtol=RTOL, atol=ATOL) crick/tests/test_stats.py:88: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = (.compare at 0x7f2a6d08dc10>, array(nan), array(0.)) kwds = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=0, atol=1e-08', 'verbose': True} @wraps(func) def inner(*args, **kwds): with self._recreate_cm(): > return func(*args, **kwds) E AssertionError: E Not equal to tolerance rtol=0, atol=1e-08 E E x and y nan location mismatch: E x: array(nan) E y: array(0.) /usr/lib64/python3.8/contextlib.py:75: AssertionError __________________________ test_kurt[False-False-x2] ___________________________ x = array([1]), bias = False, fisher = False @pytest.mark.parametrize( "x", [ normal, empty, one, duplicate, different, ], ) @pytest.mark.parametrize("bias", [True, False]) @pytest.mark.parametrize("fisher", [True, False]) def test_kurt(x, bias, fisher): stats = pytest.importorskip("scipy.stats") s = SummaryStats() s.update(x) res = s.kurt(bias=bias, fisher=fisher) if len(x): sol = stats.kurtosis(x[~np.isnan(x)], bias=bias, fisher=fisher) else: sol = np.nan > np.testing.assert_allclose(res, sol, rtol=RTOL, atol=ATOL) crick/tests/test_stats.py:88: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = (.compare at 0x7f2a6a511940>, array(nan), array(0.)) kwds = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=0, atol=1e-08', 'verbose': True} @wraps(func) def inner(*args, **kwds): with self._recreate_cm(): > return func(*args, **kwds) E AssertionError: E Not equal to tolerance rtol=0, atol=1e-08 E E x and y nan location mismatch: E x: array(nan) E y: array(0.) /usr/lib64/python3.8/contextlib.py:75: AssertionError __________________________ test_kurt[False-False-x3] ___________________________ x = array([1, 1]), bias = False, fisher = False @pytest.mark.parametrize( "x", [ normal, empty, one, duplicate, different, ], ) @pytest.mark.parametrize("bias", [True, False]) @pytest.mark.parametrize("fisher", [True, False]) def test_kurt(x, bias, fisher): stats = pytest.importorskip("scipy.stats") s = SummaryStats() s.update(x) res = s.kurt(bias=bias, fisher=fisher) if len(x): sol = stats.kurtosis(x[~np.isnan(x)], bias=bias, fisher=fisher) else: sol = np.nan > np.testing.assert_allclose(res, sol, rtol=RTOL, atol=ATOL) crick/tests/test_stats.py:88: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = (.compare at 0x7f2a6d079f70>, array(nan), array(0.)) kwds = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=0, atol=1e-08', 'verbose': True} @wraps(func) def inner(*args, **kwds): with self._recreate_cm(): > return func(*args, **kwds) E AssertionError: E Not equal to tolerance rtol=0, atol=1e-08 E E x and y nan location mismatch: E x: array(nan) E y: array(0.) /usr/lib64/python3.8/contextlib.py:75: AssertionError ============================= slowest 10 durations ============================= 0.27s call crick/tests/test_stats.py::test_skew[True-x0] 0.06s call crick/tests/test_tdigest.py::test_merge 0.03s call crick/tests/test_tdigest.py::test_distributions[data0] 0.03s call crick/tests/test_tdigest.py::test_distributions[data2] 0.03s call crick/tests/test_tdigest.py::test_distributions[data5] 0.03s call crick/tests/test_tdigest.py::test_distributions[data1] 0.03s call crick/tests/test_tdigest.py::test_distributions[data6] 0.02s call crick/tests/test_tdigest.py::test_distributions[data7] 0.02s call crick/tests/test_tdigest.py::test_distributions[data4] 0.02s call crick/tests/test_tdigest.py::test_distributions[data3] =========================== short test summary info ============================ FAILED crick/tests/test_stats.py::test_skew[True-x2] - AssertionError: FAILED crick/tests/test_stats.py::test_skew[True-x3] - AssertionError: FAILED crick/tests/test_stats.py::test_skew[False-x2] - AssertionError: FAILED crick/tests/test_stats.py::test_skew[False-x3] - AssertionError: FAILED crick/tests/test_stats.py::test_kurt[True-True-x2] - AssertionError: FAILED crick/tests/test_stats.py::test_kurt[True-True-x3] - AssertionError: FAILED crick/tests/test_stats.py::test_kurt[True-False-x2] - AssertionError: FAILED crick/tests/test_stats.py::test_kurt[True-False-x3] - AssertionError: FAILED crick/tests/test_stats.py::test_kurt[False-True-x2] - AssertionError: FAILED crick/tests/test_stats.py::test_kurt[False-True-x3] - AssertionError: FAILED crick/tests/test_stats.py::test_kurt[False-False-x2] - AssertionError: FAILED crick/tests/test_stats.py::test_kurt[False-False-x3] - AssertionError: ======================== 12 failed, 85 passed in 1.22s ========================= ```

Environment:

List of installed modules in build env: ```console Package Version ------------------ ------- build 1.0.3 cppclean 0.13 Cython 3.0.6 distro 1.8.0 dnf 4.18.2 exceptiongroup 1.1.3 gpg 1.23.2 importlib-metadata 7.0.0 iniconfig 2.0.0 installer 0.7.0 libdnf 0.72.0 numpy 1.24.4 packaging 23.2 pluggy 1.3.0 pyproject_hooks 1.0.0 pytest 7.4.3 python-dateutil 2.8.2 SciPy 1.8.1 setuptools 69.0.2 setuptools-scm 8.0.4 six 1.16.0 tomli 2.0.1 typing_extensions 4.9.0 versioneer 0.29 wheel 0.42.0 zipp 3.17.0 ```
kloczek commented 7 months ago

The same units have been failing with 0.0.4.

milesgranger commented 7 months ago

Hi @kloczek thanks for the detailed report!

I was able to replicate it, but confused why it is passing on CI.

Found updating the requirements you're using to set the following makes it work. numpy==1.26.2 scipy==1.11.4

CI is unpinned and thus installing the latest so likely explains that bit. I'll close for now, but please reopen if needed.

kloczek commented 7 months ago

Found updating the requirements you're using to set the following makes it work. numpy==1.26.2 scipy==1.11.4

As I'm still on python 3.8.x I cannot update those two modules (they are only for python >= 3.9). 😞 I'll add note to recheck those unit when all in my distro will be moved to python >= 3.9. Thank you fot reply and your time 👍 😄

milesgranger commented 7 months ago

Ah ya, Okay I missed that. I replicated it on 3.8 and thought I'd try to replicate on 3.9 as well which I could. Then only went on further from that environment to find the 'solution'. thanks again!