search

conda-forge / netcdf4-feedstock

A conda-smithy repository for netcdf4.
BSD 3-Clause "New" or "Revised" License
3 stars 25 forks source link

run all tests #142

Closed ocefpaf closed 2 months ago

ocefpaf commented 1 year ago

This is something we should've done a long time ago.

conda-forge-linter commented 1 year ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

ocefpaf commented 1 year ago

There are 8 tests failing on ppc64le:

ERROR: runTest (tst_multifile.NonuniformTimeTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/test_tmp/test/tst_multifile.py", line 126, in runTest
    dates.extend(cftime.num2date(t[:], t.units, calendar))
  File "src/cftime/_cftime.pyx", line 586, in cftime._cftime.num2date
  File "src/cftime/_cftime.pyx", line 385, in cftime._cftime.cast_to_int
OverflowError: time values outside range of 64 bit signed integers

======================================================================
ERROR: runTest (tst_multifile2.NonuniformTimeTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/test_tmp/test/tst_multifile2.py", line 114, in runTest
    dates.extend(cftime.num2date(t[:], t.units, t.calendar))
  File "src/cftime/_cftime.pyx", line 586, in cftime._cftime.num2date
  File "src/cftime/_cftime.pyx", line 385, in cftime._cftime.cast_to_int
OverflowError: time values outside range of 64 bit signed integers

======================================================================
FAIL: runTest (tst_vlen.Vlen_ScaledInts)
testing packing float vlens as scaled integers (issue #1003).
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/test_tmp/test/tst_vlen.py", line 220, in runTest
    assert(err.max() < nc['vl'].scale_factor)
AssertionError

======================================================================
FAIL: runTest (tst_masked.PrimitiveTypesTestCase)
testing auto-conversion of masked arrays and packed integers
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/test_tmp/test/tst_masked.py", line 118, in runTest
    assert_array_equal(datapacked[:],packeddata2)
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 934, in assert_array_equal
    assert_array_compare(operator.__eq__, x, y, err_msg=err_msg,
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 844, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not equal

Mismatched elements: 2 / 10 (20%)
Max absolute difference: 20271
Max relative difference: 1.54375143
 x: array([-32134,  30448,   5655,   3599,      0,      0,      0,      0,
       -27211, -32766], dtype=int16)
 y: array([ 13131,  19120,   5655,   3599,      0,      0,      0,      0,
       -27211, -32766], dtype=int16)

======================================================================
FAIL: runTest (tst_masked2.PrimitiveTypesTestCase)
testing auto-conversion of masked arrays and packed integers
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/test_tmp/test/tst_masked2.py", line 118, in runTest
    assert_array_almost_equal(data,datacheck4)
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 1046, in assert_array_almost_equal
    assert_array_compare(compare, x, y, err_msg=err_msg, verbose=verbose,
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 768, in assert_array_compare
    flagged = func_assert_same_pos(x, y, func=isnan, hasval='nan')
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 745, in func_assert_same_pos
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 6 decimals

x and y nan location mismatch:
 x: MaskedArray([nan, nan,  0.,  0.], dtype=float32)
 y: MaskedArray([1.5625,    inf, 3.75  , 4.125 ], dtype=float32)

======================================================================
FAIL: runTest (tst_multifile.VariablesTestCase)
testing multi-file dataset access
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/test_tmp/test/tst_multifile.py", line 52, in runTest
    assert_array_equal(np.arange(0,nx),f.variables['x'][:])
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 934, in assert_array_equal
    assert_array_compare(operator.__eq__, x, y, err_msg=err_msg,
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 844, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not equal

Mismatched elements: 79 / 100 (79%)
Max absolute difference: 471670300
Max relative difference: 97.
 x: array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,
       17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,
       34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50,...
 y: array([        0,         0, 202182159, 471670303,         0,         0,
              -1,  16777215,         8,         9,         0,         0,
               3,         5,         0,         0,        -1,  16777215,...

======================================================================
FAIL: runTest (tst_multifile2.VariablesTestCase)
testing multi-file dataset access
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/test_tmp/test/tst_multifile2.py", line 52, in runTest
    assert_array_equal(np.arange(0,nx),f.variables['x'][:])
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 934, in assert_array_equal
    assert_array_compare(operator.__eq__, x, y, err_msg=err_msg,
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 844, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not equal

Mismatched elements: 80 / 100 (80%)
Max absolute difference: 812388678
Max relative difference: 97.
 x: array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,
       17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,
       34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50,...
 y: array([415223568,  44201712,  67438087, 202182159,         0,         0,
              -1,  16777215,         8,         9, 812388688,  44201712,
               8,         6,         0,         0,        -1,  16777215,...

======================================================================
FAIL: test_unmasked (tst_scaled.SetAutoScaleTrue)
Testing auto-scaling of variables for set_auto_scale(True)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/test_tmp/test/tst_scaled.py", line 116, in test_unmasked
    assert_array_almost_equal(vv_scaled, np.ones(10))
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 1046, in assert_array_almost_equal
    assert_array_compare(compare, x, y, err_msg=err_msg, verbose=verbose,
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 844, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 6 decimals

Mismatched elements: 8 / 10 (80%)
Max absolute difference: 32135.
Max relative difference: 32135.
 x: MaskedArray([-3.2134e+04,  3.0448e+04,  5.6550e+03,  3.5990e+03,
              0.0000e+00,  0.0000e+00,  0.0000e+00,  0.0000e+00,
              1.0000e+00,  1.0000e+00], dtype=float32)
 y: array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1.])

======================================================================
FAIL: test_auto_scale_write (tst_scaled.WriteAutoScaleTest)
Testing automatic packing to all kinds of integer types
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/test_tmp/test/tst_scaled.py", line 190, in test_auto_scale_write
    assert(maxerrnorm < 0.5 + 1e-5)
AssertionError

======================================================================
FAIL: runTest (tst_types.PrimitiveTypesTestCase)
testing primitive data type
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/test_tmp/test/tst_types.py", line 69, in runTest
    assert_array_almost_equal(datarr,ranarr[1:n1dim].astype(data.dtype))
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 1046, in assert_array_almost_equal
    assert_array_compare(compare, x, y, err_msg=err_msg, verbose=verbose,
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 768, in assert_array_compare
    flagged = func_assert_same_pos(x, y, func=isnan, hasval='nan')
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 745, in func_assert_same_pos
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 6 decimals

x and y nan location mismatch:
 x: array([[ 2.350989e-38,           nan,  3.030902e+24,  7.134550e+31,
                  nan,           nan,  8.923098e-32,  3.500230e-27,
        -2.458268e-38,  1.775738e+36],...
 y: array([[ 2.350989e-38,           nan,  1.485376e-43,           nan,
                  nan,           nan,  8.923098e-32,  3.500230e-27,
        -2.458268e-38,  1.775738e+36],...

----------------------------------------------------------------------
Ran 97 tests in 109.403s

FAILED (failures=8, errors=2, skipped=1)

I don't think that ppc64le is used by many or if it is even support upstream. @conda-forge/netcdf4 team should we try to fix these or just skip?

xylar commented 1 year ago

@ocefpaf, I'm afraid I don't know about whether the tests are expected to pass on ppc64le. I do know that we're quite interested in having conda-forge package support on this architecture because some of the newer HPC systems use it and our testing software can't currently be supported.

@dopplershift, any insight into whether we would expect these tests to pass?

dopplershift commented 1 year ago

@WardF Any insights on testing on ppc64le?

ocefpaf commented 1 year ago

I do know that we're quite interested in having conda-forge package support on this architecture because some of the newer HPC systems

Good to know!

WardF commented 1 year ago

@dopplershift I would expect the libnetcdf tests to pass on ppc64le (and indeed they do, over at libnetcdf-feedstock). I would have to dig into the failures observed here to see if they are occurring at the C library level (in which case I can probably help) or somewhere in the Python code (in which case I am less equipped to help). But to answer the specific question, ppc64le is one of the platforms we support, and if something is broken in the libnetcdf library, we would need to fix it.

WardF commented 1 year ago

Thinking more about this, it is possible that the python testing is covering code that's not being tested in libnetcdf-feedstock or in the netcdf-c code. If somebody more knowledgeable about Python can make the determination that the issue is the C library, I'm happy to jump in and 1) expand our testing and 2) fix the underlying code so that the tests pass.

xylar commented 1 year ago

I went ahead and posted an issue: https://github.com/Unidata/netcdf4-python/issues/1196

@jswhit, as the main developer I'm wondering if you would be willing to comment either there or here.

jswhit commented 1 year ago

so this is a new thing (testing on ppc64le) and not something that used to pass and now doesn't?

xylar commented 1 year ago

@jswhit, yes, that's right. Some folks ran into trouble with the last release (see #141) and @ocefpaf decided to turn on a lot more testing, presumably to see if there were any obvious problems with the conda-forge build. In that process, everything worked great except ppc64le). So these tests have never been run before by us. If we were to ignore them, that would be "business as usual" for us. But we're trying to be more diligent than that :-)

jswhit commented 1 year ago

some of the test failures are for cftime and appear to be related to be overflowing the range of 64 bit integers when casting from a float. I wonder if it could be a problem with the version of numpy being used.

xylar commented 1 year ago

I wonder if it could be a problem with the version of numpy being used.

The ppc64le build is using numpy 1.20 (1.21 for python 3.10), the same as the other linux builds that pass. So it doesn't seem obvious to me that the numpy version could explain any differences unless there is a problem with the ppc64le build of numpy 1.20 that isn't in the others.

jswhit commented 1 year ago

I wonder if it could be a problem with the version of numpy being used.

The ppc64le build is using numpy 1.20 (1.21 for python 3.10), the same as the other linux builds that.So it doesn't seem obvious to me that the numpy version could explain any differences unless there is a problem with the ppc64le build of numpy 1.20 that isn't in the others.

That's just a wild guess - how hard would it be to try numpy 1.22 or 1.23?

xylar commented 1 year ago

That's just a wild guess - how hard would it be to try numpy 1.22 or 1.23?

I'll look into doing that for testing purposes but part of what makes conda-forge work is that we have specific versions of packages like numpy that we use consistently across the ecosystem of packages. So if that's a requirement, we'd need to disable ppc64le support until we're ready to migrate to a newer numpy.

xylar commented 1 year ago

@ocefpaf, if this works, maybe it would be possible to pin a higher version of numpy? According to the knowledge base, this is okay but what would be downstream consequences be? Also, could we pin just for ppc64le? I'm almost certainly jumping the gun since we don't know if it will work...

xylar commented 1 year ago

No luck with numpy 1.22. let's try 1.23.

xylar commented 1 year ago

@jswhit, I appreciate the recommendation but no luck there. Let us know if anything else comes to mind.

ocefpaf commented 1 year ago

BTW, we do run all the tests in the cftime feedstock and we do not catch that error. It is possible that the tests do not cover this problem.

xylar commented 1 year ago

@ocefpaf, I think it's better at this point to just disable the tests for ppc64le as you suggested earlier. It seems better to have the tests for other architectures than not at all. If we or someone else has a chance to investigate the failures at a later time, that would be great! But we have to be realistic about our limited time right now.

ocefpaf commented 1 year ago

@xylar we don't need new artifacts to be built, so we can wait for a new release and just rebase here. I'm OK keeping this as a draft until then.

xylar commented 1 year ago

@conda-forge-admin, please rerender

xylar commented 1 year ago

@ocefpaf, do we want to explore this again following #146?

github-actions[bot] commented 1 year ago

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/netcdf4-feedstock/actions/runs/3487394746.

ocefpaf commented 1 year ago

@ocefpaf, do we want to explore this again following #146?

Rebased and restarted. Let's see how it goes.

xylar commented 1 year ago

Windows tests are passing, so that's the one that I was concerned about. the alternative linux architectures are clearly still having trouble.

It's frustrating that so many folks are having trouble with DLLs in the windows version but I don't have any relevant expertise to help there or to debug with folks if it's just user error.

ocefpaf commented 1 year ago

It's frustrating that so many folks are having trouble with DLLs in the windows version but I don't have any relevant expertise to help there or to debug with folks if it's just user error.

Are folks reporting to you directly these failures? Are those with the conda-forge package? Sorry if I missed any important ping, I'm quite slow lately with my notifications.

xylar commented 1 year ago

I'm thinking of the comments on #136 that keep coming in and then #145 just posted yesterday or the day before.

xylar commented 1 year ago

I don't work directly with anyone using Windows as far as I'm aware.

ocefpaf commented 1 year ago

I don't work directly with anyone using Windows as far as I'm aware.

Lucky you!

I'm thinking of the comments on #136 that keep coming in and then #145 just posted yesterday or the day before.

They are unrelated to netcdf4 per-se. I commented on both issues.

xylar commented 1 year ago

@ocefpaf, great I'm glad you were able to help!

conda-forge-webservices[bot] commented 2 months ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

ocefpaf commented 2 months ago

@xylar it seems that all the tests are passing now 🎉

Do you mind taking a look if I for it right?

xylar commented 2 months ago

I'm a bit rusty on the details of this PR but everything looks good to me!

xylar commented 2 months ago

It looks like the packages for build 0 are done so I'm going to merge this.