search

conda-forge / numba-feedstock

A conda-smithy repository for numba.
BSD 3-Clause "New" or "Revised" License
0 stars 27 forks source link

Test failures on Windows Python 3.5 and 3.6 64-bit #3

Closed jakirkham closed 4 years ago

jakirkham commented 6 years ago 
======================================================================
FAIL: test_linalg_cond (numba.tests.test_linalg.TestLinalgCond)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\bld\numba_1524336824076\_test_env\lib\site-packages\numba\tests\test_linalg.py", line 1936, in test_linalg_cond
    check(a, p=p)
  File "C:\bld\numba_1524336824076\_test_env\lib\site-packages\numba\tests\test_linalg.py", line 1923, in check
    np.testing.assert_allclose(got, expected, rtol=resolution)
  File "C:\bld\numba_1524336824076\_test_env\lib\site-packages\numpy\testing\nose_tools\utils.py", line 1396, in assert_allclose
    verbose=verbose, header=header, equal_nan=equal_nan)
  File "C:\bld\numba_1524336824076\_test_env\lib\site-packages\numpy\testing\nose_tools\utils.py", line 779, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=5e-15, atol=0
(mismatch 100.0%)
 x: array(6.148634)
 y: array(8.508103)
======================================================================
FAIL: test_linalg_det (numba.tests.test_linalg.TestLinalgDetAndSlogdet)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\bld\numba_1524336824076\_test_env\lib\site-packages\numba\tests\test_linalg.py", line 1781, in test_linalg_det
    self.do_test("det", self.check_det, cfunc)
  File "C:\bld\numba_1524336824076\_test_env\lib\site-packages\numba\tests\test_linalg.py", line 1758, in do_test
    check(cfunc, a)
  File "C:\bld\numba_1524336824076\_test_env\lib\site-packages\numba\tests\test_linalg.py", line 1707, in check_det
    np.testing.assert_allclose(got, expected, rtol=resolution)
  File "C:\bld\numba_1524336824076\_test_env\lib\site-packages\numpy\testing\nose_tools\utils.py", line 1396, in assert_allclose
    verbose=verbose, header=header, equal_nan=equal_nan)
  File "C:\bld\numba_1524336824076\_test_env\lib\site-packages\numpy\testing\nose_tools\utils.py", line 779, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=5e-15, atol=0
(mismatch 100.0%)
 x: array(0.704632-0.709573j)
 y: array(1.034743-0.852443j)
======================================================================
FAIL: test_linalg_slogdet (numba.tests.test_linalg.TestLinalgDetAndSlogdet)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\bld\numba_1524336824076\_test_env\lib\site-packages\numba\tests\test_linalg.py", line 1786, in test_linalg_slogdet
    self.do_test("slogdet", self.check_slogdet, cfunc)
  File "C:\bld\numba_1524336824076\_test_env\lib\site-packages\numba\tests\test_linalg.py", line 1758, in do_test
    check(cfunc, a)
  File "C:\bld\numba_1524336824076\_test_env\lib\site-packages\numba\tests\test_linalg.py", line 1737, in check_slogdet
    got_conv, expected[0], nulp=10)
  File "C:\bld\numba_1524336824076\_test_env\lib\site-packages\numpy\testing\nose_tools\utils.py", line 1459, in assert_array_almost_equal_nulp
    raise AssertionError(msg)
AssertionError: X and Y are not equal to 10 ULP
======================================================================
FAIL: test_linalg_matrix_power (numba.tests.test_linalg.TestLinalgMatrixPower)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\bld\numba_1524336824076\_test_env\lib\site-packages\numba\tests\test_linalg.py", line 2115, in test_linalg_matrix_power
    check(a, pwr)
  File "C:\bld\numba_1524336824076\_test_env\lib\site-packages\numba\tests\test_linalg.py", line 2103, in check
    np.testing.assert_allclose(got, expected, rtol=res, atol=res)
  File "C:\bld\numba_1524336824076\_test_env\lib\site-packages\numpy\testing\nose_tools\utils.py", line 1396, in assert_allclose
    verbose=verbose, header=header, equal_nan=equal_nan)
  File "C:\bld\numba_1524336824076\_test_env\lib\site-packages\numpy\testing\nose_tools\utils.py", line 779, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=5e-15, atol=5e-15
(mismatch 100.0%)
 x: array([[ 0.131856-0.494001j,  0.110899+0.219773j, -0.317231+0.198697j,
        -0.120423+0.128223j, -0.099134+0.575547j, -0.123573+0.178233j,
        -0.317548+0.134027j],...
 y: array([[ -65.911595 -61.460872j,  -57.438371 -14.088685j,
         -78.757699  -9.484391j,  -31.219762 -19.690842j,
         -38.113793  -9.252879j,  -17.272821 +71.826947j,...

ref: https://ci.appveyor.com/project/conda-forge/numba-feedstock/build/1.0.1/job/al8tbhrvio3ogvhq ref: https://ci.appveyor.com/project/conda-forge/numba-feedstock/build/1.0.1/job/6tcdjqtueu3meg2q

jakirkham commented 6 years ago

Test failures on Windows continue with Numba 0.38.0. Though there cause is different. Snippet of the failures seen on Python 3.6. The Python 3.5 log was vague on what failed

======================================================================
FAIL: test_expm1_D_D (numba.tests.test_ufuncs.TestLoopTypesComplexNoPython)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\bld\numba_1525059992876\_test_env\lib\site-packages\numba\tests\test_ufuncs.py", line 1515, in test_template
    self._check_loop(fn, ufunc, loop)
  File "C:\bld\numba_1525059992876\_test_env\lib\site-packages\numba\tests\test_ufuncs.py", line 1472, in _check_loop
    self._check_ufunc_with_dtypes(fn, ufunc, letter_types)
  File "C:\bld\numba_1525059992876\_test_env\lib\site-packages\numba\tests\test_ufuncs.py", line 1501, in _check_ufunc_with_dtypes
    ulps=ulps)
  File "C:\bld\numba_1525059992876\_test_env\lib\site-packages\numba\tests\support.py", line 257, in assertPreciseEqual
    self.fail("when comparing %s and %s: %s" % (first, second, failure_msg))
AssertionError: when comparing [-0.45969769-0.84147098j         nan       +nanj -0.45969769-0.84147098j
  0.        +0.j         -0.68297786+4.47046238j         nan       +nanj
 -0.68297786+4.47046238j  0.        +0.j        ] and [-0.80123389-0.30955988j         nan       +nanj -0.80123389-0.30955988j
  0.        +0.j         -0.68297786+4.47046238j         nan       +nanj
 -0.68297786+4.47046238j  0.        +0.j        ]: -0.45969769413186023 != -0.801233889653587 within 2.018169754409307e-16 delta : ufunc 'expm1' arrays differ ([array([0. -1.j , nan+nanj, 0. -1.j , 0. +0.j , 1.5+1.5j, nan+nanj,
       1.5+1.5j, 0. +0.j ]), array([-0.80123389-0.30955988j,         nan       +nanj,
       -0.80123389-0.30955988j,  0.        +0.j        ,
       -0.68297786+4.47046238j,         nan       +nanj,
       -0.68297786+4.47046238j,  0.        +0.j        ])]):
args: double
expected [-0.45969769-0.84147098j         nan       +nanj -0.45969769-0.84147098j
  0.        +0.j         -0.68297786+4.47046238j         nan       +nanj
 -0.68297786+4.47046238j  0.        +0.j        ]
got [-0.80123389-0.30955988j         nan       +nanj -0.80123389-0.30955988j
  0.        +0.j         -0.68297786+4.47046238j         nan       +nanj
 -0.68297786+4.47046238j  0.        +0.j        ]
======================================================================
FAIL: test_expm1_F_F (numba.tests.test_ufuncs.TestLoopTypesComplexNoPython)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\bld\numba_1525059992876\_test_env\lib\site-packages\numba\tests\test_ufuncs.py", line 1515, in test_template
    self._check_loop(fn, ufunc, loop)
  File "C:\bld\numba_1525059992876\_test_env\lib\site-packages\numba\tests\test_ufuncs.py", line 1472, in _check_loop
    self._check_ufunc_with_dtypes(fn, ufunc, letter_types)
  File "C:\bld\numba_1525059992876\_test_env\lib\site-packages\numba\tests\test_ufuncs.py", line 1501, in _check_ufunc_with_dtypes
    ulps=ulps)
  File "C:\bld\numba_1525059992876\_test_env\lib\site-packages\numba\tests\support.py", line 257, in assertPreciseEqual
    self.fail("when comparing %s and %s: %s" % (first, second, failure_msg))
AssertionError: when comparing [-0.45969772-0.84147096j         nan       +nanj -0.45969772-0.84147096j
  0.        +0.j         -0.6829779 +4.4704623j          nan       +nanj
 -0.6829779 +4.4704623j   0.        +0.j        ] and [-0.8012339-0.3095599j        nan      +nanj -0.8012339-0.3095599j
  0.       +0.j        -0.6829779+4.4704623j        nan      +nanj
 -0.6829779+4.4704623j  0.       +0.j       ]: -0.45969772 != -0.8012339 within 1.0834966701622761e-07 delta : ufunc 'expm1' arrays differ ([array([0. -1.j , nan+nanj, 0. -1.j , 0. +0.j , 1.5+1.5j, nan+nanj,
       1.5+1.5j, 0. +0.j ], dtype=complex64), array([-0.8012339-0.3095599j,        nan      +nanj,
       -0.8012339-0.3095599j,  0.       +0.j       ,
       -0.6829779+4.4704623j,        nan      +nanj,
       -0.6829779+4.4704623j,  0.       +0.j       ], dtype=complex64)]):
args: single
expected [-0.45969772-0.84147096j         nan       +nanj -0.45969772-0.84147096j
  0.        +0.j         -0.6829779 +4.4704623j          nan       +nanj
 -0.6829779 +4.4704623j   0.        +0.j        ]
got [-0.8012339-0.3095599j        nan      +nanj -0.8012339-0.3095599j
  0.       +0.j        -0.6829779+4.4704623j        nan      +nanj
 -0.6829779+4.4704623j  0.       +0.j       ]
----------------------------------------------------------------------
Ran 6793 tests in 3697.492s
FAILED (failures=7, skipped=75, expected failures=6)

ref: https://ci.appveyor.com/project/conda-forge/numba-feedstock/build/1.0.10/job/dq4jkmea4wao4ci5 ref: https://ci.appveyor.com/project/conda-forge/numba-feedstock/build/1.0.10/job/tbt52ea0yarvyjk5

jakirkham commented 6 years ago

Raised upstream as issue ( https://github.com/numba/numba/issues/2925 ).

jakirkham commented 6 years ago

Made a note about the Windows 64-bit Python 3.5 failure in this comment.

flutefreak7 commented 5 years ago

Is there any roadmap to getting conda-forge numba to work for Windows? Have the tests been attempted on 3.7? Have they been attempted with recent versions of numba? There have been over 20 releases since this issue.

This is one of the things that makes me hesitant about attempting to create a fully conda-forge based environment since I have numba as a dependency. I previously ran into issues mixing conda and pip, and I have read that there are clear issues with mixing conda-forge with the defaults channel. Because of these issues I'm using pip only right now but trying to figure out whether I can use conda or if I have to stick with pip until issues like this are resolved.

I also maybe need help understanding how conda builds and tests work. So does conda-forge require that all tests pass for a build to be allowed? I assume that these versions of numba are working fine for everyone else via pip and the conda defaults, so I don't understand why conda-forge doesn't trust the code? Is it a compiling/packaging issue on the conda-forge side or simply a testing/diagnostics issue? Is there a way to allow the package to compile for windows users and simply document the testing anomaly? Shift the risk to the user?

Thank you!

isuruf commented 5 years ago

So does conda-forge require that all tests pass for a build to be allowed? I assume that these versions of numba are working fine for everyone else via pip and the conda defaults, so I don't understand why conda-forge doesn't trust the code?

For compiled code we usually run the full test suite to make sure the packages works fine. Even if the same version works for everyone else via pip and conda defaults, this conda-forge package might not work because of other factors. For eg: if llvm was compiled without some require patches. So we prefer to get the tests working.

Can you please send a PR to enable windows and see whether tests pass? We can ask for help from numba devs to make the tests pass.

flutefreak7 commented 5 years ago

I'd love to help, but submitting the PR to enable Windows is a bit out of my league. Thank you for your response and anything you're able to do to help get numba working on conda-forge for Windows users!

I posted a comment on the upstream numba issue. They filed it as a "heisenbug" (lol) so it was unlikely to get high visibility.

isuruf commented 5 years ago

I'd love to help, but submitting the PR to enable Windows is a bit out of my league.

It's not that big of a deal. Remove the line at https://github.com/conda-forge/numba-feedstock/blob/master/recipe/meta.yaml#L16 and send a PR

mingwandroid commented 4 years ago

and I have read that there are clear issues with mixing conda-forge with the defaults channel

There used to be, but since January this year that's no longer been a problem. I mix and match packages from both defaults and conda-forge all day every day (except when I'm alseep or eating!) without issue. This is because we aligned on our compiler stack and fixed how we handle pinning.

mingwandroid commented 4 years ago

You can have a nearly-all-conda-forge env with just single packages from defaults, provided the deps are built and built well. conda install -c conda-forge <lots-of-stuff> defaults::numba

jakirkham commented 4 years ago

Closing this out as we are building numba on Windows.