directory discovery confused

[jayvdb]

For https://github.com/wikimedia/pywikibot-core , specifying the directory tests/ causes only tests/tests_tests.py to be run.

$ green tests
tests: max_retries reduced from 25 to 1
ERROR: HttpServerProblemTestCase: accessing http://httpbin.org/status/502 caused exception:
ERROR: HTTP status: 502
Traceback (most recent call last):
  File ".../tests/aspects.py", line 505, in setUpClass
    raise ServerError('HTTP status: %d' % r.status)
ServerError: HTTP status: 502ERROR: 
  File ".../tests/utils.py", line 84, in wrapper
    func(*args, **kwargs)
  File ".../tests/tests_tests.py", line 51, in test_assertPageTitlesEqual
    self.site)
s

Skipped tests.tests_tests.TestPageAssert.test_assertPageTitlesEqual - Test is allowed to fail.

Ran 1 test in 1.157s

OK (skips=1)

Ignore the ERRORs, Tracebacks and skips -- the above result is perfect, except that it is running 1 of 1000s of unit tests! Also running each test module works; e.g.

$ green tests.site_tests
...
Ran 34 tests in 161.793s

OK (expected_failures=2, passes=31, skips=1)

Renaming the 'tests_tests' module fixes this bug! e.g.

$ mv tests/tests_tests.py tests/_tests_tests.py
$ green tests
...

(green does fail somewhere in our test suites with an obscure message and traceback, which I'll raise as a separate bug once I've debugged it a little more)

Fwiw, I'd advise against trying to run the Pywikibot tests to understand this bug, as setting up the pywikibot tests is not fork and click. If the cause isnt immediately obvious, I can set up a simpler example of the problem, or fix it myself ;-)

[jayvdb]

fwiw, solving https://github.com/CleanCut/green/issues/87 would also solve our needs, but it seems like this bug is much smaller, and can cause problems for projects which are not using load_tests protocol.

[jayvdb]

The following is a workaround

$ green tests/*.py

[CleanCut]

As part of the Traditional feature, we lean on python's built-in unit test discovery. By default this means you are required to name your test module with the pattern test*.py for it to be discovered under a directory, as explained in the tutorial (see note 4 under External Structure).

Also following the conventions of unittest, the file pattern can also be overridden. Under green --help you will notice:

Other Options:
[...snip...]
  -p PATTERN, --file-pattern PATTERN
                        Pattern to match test files. Default is test*.py
[...snip...]

As you discovered, you can also manually specify individual files (or dotted paths that correspond to any object from a package down to a test method) which bypass discovery and load the specified target.