Package runtests.py

[mcepl]

When running test build of the new weave package (0.16.0 with tarball from PyPI) I get these errors:

[  164s] 
[  164s] ======================================================================
[  164s] ERROR: test_incorrect_ownership (test_catalog.TestIntermediateDir)
[  164s] ----------------------------------------------------------------------
[  164s] Traceback (most recent call last):
[  164s]   File "/usr/lib64/python2.7/site-packages/numpy/testing/nose_tools/decorators.py", line 147, in skipper_func
[  164s]     return f(*args, **kwargs)
[  164s]   File "/home/abuild/rpmbuild/BUILD/weave-0.16.0/weave/tests/test_catalog.py", line 69, in test_incorrect_ownership
[  164s]     raise KnownFailureTest("Can't change owner.")
[  164s] KnownFailureException: Can't change owner.
[  164s] 
[  164s] ======================================================================
[  164s] ERROR: test_char_fail (test_scxx_dict.TestDictGetItemOp)
[  164s] ----------------------------------------------------------------------
[  164s] Traceback (most recent call last):
[  164s]   File "/usr/lib64/python2.7/site-packages/numpy/testing/nose_tools/decorators.py", line 215, in knownfailer
[  164s]     raise KnownFailureException(msg)
[  164s] KnownFailureException: Test skipped due to known failure
[  164s] 
[  164s] ======================================================================
[  164s] ERROR: test_obj_fail (test_scxx_dict.TestDictGetItemOp)
[  164s] ----------------------------------------------------------------------
[  164s] Traceback (most recent call last):
[  164s]   File "/usr/lib64/python2.7/site-packages/numpy/testing/nose_tools/decorators.py", line 215, in knownfailer
[  164s]     raise KnownFailureException(msg)
[  164s] KnownFailureException: Test skipped due to known failure
[  164s] 
[  164s] ----------------------------------------------------------------------
[  164s] Ran 454 tests in 159.483s
[  164s] 
[  164s] FAILED (errors=3)
[  164s] Test failed: <unittest.runner.TextTestResult run=454 errors=3 failures=0>
[  164s] error: Test failed: <unittest.runner.TextTestResult run=454 errors=3 failures=0>
[  164s] error: Bad exit status from /var/tmp/rpm-tmp.zIP289 (%check)
[  164s] 
[  164s] 
[  164s] RPM build errors:
[  164s]     File /home/abuild/rpmbuild/SOURCES/fix_failing_tests.patch is smaller than 13 bytes
[  164s]     Bad exit status from /var/tmp/rpm-tmp.zIP289 (%check)
[  164s] 
[  164s] milic failed "build python-weave.spec" at Wed Jun 27 21:58:08 UTC 2018.
[  164s] 

Complete log is here.

[mcepl]

I need to use runtests.py, which is unfortunately not packaged in the distribution tarball available on PyPI.

[rgommers]

Those are not failures, they're errors because somehow KnownFailureException isn't recognized. How exactly do you run the tests, and what versions of nose and numpy do you have installed?

[mcepl]

When I use runtests.py it works with the same versions of nose and numpy (you can see it in the linked build.log above .. python2-nose-1.3.7-9.2 and python2-numpy-1.14.5-1.1). It's the official package for openSUSE (in the staging project, so far).

[rgommers]

There's something really weird in that log:

warning: no previously-included files matching '*.pyo' found anywhere in distribution
[    4s] writing manifest file 'weave.egg-info/SOURCES.txt'
[    4s] running build_ext
[    4s] test_simple_expr (test_ast_tools.TestHarvestVariables) ... ok
[    4s] test_fdtd_expr (test_blitz_tools.TestAstToBlitzExpr) ... ok
[    4s] test_simple_expr (test_blitz_tools.TestAstToBlitzExpr) ... ok
....
Test failed: <unittest.runner.TextTestResult run=454 errors=3 failures=0>

I don't see in the log where the tests get kicked off. And you and up with the unittest test runner. Either nosetests weave or >>> import weave; weave.test() will run the tests with nosetests rather than unittest and you will not encounter this issue.

So I suggest looking into your build+test script and figuring out how the tests get started.

[mcepl]

OK, now using plain nosetests-2.7 weave -v and result is the same. Full build.log is attached.

[mcepl]

In the end I had to add this patch to make tests passing with plain nosetests -v weave runner. http://paste.opensuse.org/94584272

[pv]

runtest.py simply runs "import weave; weave.test()". This is not the same thing as "nosetests weave". The latter iirc does not register the "knownfailure" nose plugin from numpy.testing.

[pv]

I would expect "python setup.py test" be similar to "nosetest" i.e. it does not register the necessary test plugin.

[pv]

This is common to all projects using numpy.testing test runner --- the tests should be launched via "import somemodule; somemodule.test()" and not by nosetests.

[mcepl]

The latter iirc does not register the "knownfailure" nose plugin from numpy.testing.

Well, it would be good, if the requirement for knownfailure nose plugin was mentioned in README or something.

[mcepl]

OK, it is fascinating:

/usr/bin/python2 -c 'import weave; weave.test(verbose=2)'

makes tests passing, but it would be really lovely, if this information was found somewhere (README?).

And of course it looks like that runtests.py file is not needed at all in the end.

[rgommers]

Sorry forgot that nosetests weave was different. Fixed now in the README. Thanks @mcepl, @pv