Currently (v2.4.1) green creates an empty test suite if no tests are found and then runs that empty suite, returning passing code. This can cause false-positives for CI and should be changed.
Example:
$ python -m green -vvv -s 1 -W -t --run-coverage emap-convert --debug
2016-05-24 15:32:20 DEBUG Attempting to load target 'emap-convert' with file_pattern 'test*.py'
2016-05-24 15:32:20 DEBUG IGNORED exception: No module named 'emap-convert'
2016-05-24 15:32:20 DEBUG Found 0 tests for target 'emap-convert'
2016-05-24 15:32:20 DEBUG No test loading attempts succeeded. Created an empty test suite.
Coverage.py warning: No data was collected.
Green 2.4.1, Coverage 4.1, Python 3.4.4
Name Stmts Miss Cover
---------------------------
Ran 0 tests in 0.280s
No Tests Found
Build succeeded
As you can see, this technically failed because the package/module name was incorrect (it should have been emap_convert - note the underscore). However, Green reported a pass.
Discussion:
I would recommend changing the default action to fail if no tests are found, rather than pass. This might cause issues with some (albeit very odd) usage cases, so perhaps it's prudent to add a CLI option --empty-suite-passes or something like that.
Summary:
Currently (v2.4.1)
green
creates an empty test suite if no tests are found and then runs that empty suite, returning passing code. This can cause false-positives for CI and should be changed.Example:
As you can see, this technically failed because the package/module name was incorrect (it should have been
emap_convert
- note the underscore). However, Green reported a pass.Discussion:
I would recommend changing the default action to fail if no tests are found, rather than pass. This might cause issues with some (albeit very odd) usage cases, so perhaps it's prudent to add a CLI option
--empty-suite-passes
or something like that.