Subpackage support?

[gforcada]

FEATURE REQUEST

It would be nice, like pytest does for example, to be able to place tests not only on a single flat package, but organize them in subpackages ✨ 🍀

What I did:

I created a subpackage inside my tests package on a plone add-on and tried to run the test module inside that subpackage, but the test runner reported no tests found 🤷🏾 😅

What I expect to happen:

That zope.testrunner finds test modules within subpackages.

What actually happened:

No tests on the subpackages were found.

What version of Python and Zope/Addons I am using:

• Fedora Linux 37
• Python 3.8
• zope.testrunner 5.4.0

[d-maurer]

Gil Forcada Codinachs wrote at 2023-1-14 05:51 -0800:

... It would be nice, like pytest does for example, to be able to place tests not only on a single flat package, but organize them in subpackages ? ?

I believe this is already possible and can be achieve in the following ways:

1. via the -s | --package option.

   Likely, this is not yet what you want, as it will run all tests found in the [sub]package (while you likely want to run only a subset)

2. via the -m | --module option.

   This restricts the modules (including packages, subpackages) from which tests are taken. The option value is a regex used in "search mode" (rather than "match mode"). It is applied to the fully qualified module name; this way, you can select test subsets.

3. via the -t | --test option.

   The restricts the run tests based on their name and class. The option value is a regex used in "search mode". It is applied to <test name> (<fully qualified class name>). If your reqex targets the class name part, you can choose a subset of tests definied in a (test) subpackage.

[gforcada]

@d-maurer 🤔 then maybe is something wrong with my setup, as I run it via -s and -t and no test is found.

My structure is:

src/
├── my
│   ├── package
│   │   ├── tests
│   │   │   ├── subpackage
│   │   │   │   ├── __init__.py
│   │   │   │   ├── test_that_NOT_runs.py
│   │   │   ├── __init__.py
│   │   │   ├── test_that_runs.py

So when I do run bin/test -s my.package the tests on test_that_runs.py do get picked up and run, but the ones on test_that_NOT_runs.py do not get found.

Both __init__.py are empty, if that matters 🤷🏾

[d-maurer]

Gil Forcada Codinachs wrote at 2023-1-14 09:01 -0800:

@d-maurer ? then maybe is something wrong with my setup, as I run it via -s and -t and no test is found.

My structure is:

src/
├── my
│   ├── package
│   │   ├── tests
│   │   │   ├── subpackage
│   │   │   │   ├── __init__.py
│   │   │   │   ├── test_that_NOT_runs.py
│   │   │   ├── __init__.py
│   │   │   ├── test_that_runs.py

So when I do run bin/test -s my.package the tests on test_that_runs.py do get picked up and run, but the ones on test_that_NOT_runs.py do not get found.

You could try to put the test_that_NOT_runs.py into into a tests subpackage. The test loader likely only looks for test modules in those subpackages.

Alternatively, you use tell it via an option to also look elsewhere. I believe that this is controlled via the tests-pattern option.

[gforcada]

That's exactly what I wanted to avoid, having all files in the same package, on a few distributions we have tens of modules, which makes it difficult to handle.

I will have a look at the tests-pattern option 👍🏾

[d-maurer]

Gil Forcada Codinachs wrote at 2023-1-14 09:34 -0800:

That's exactly what I wanted to avoid, having all files in the same package, on a few distributions we have tens of modules, which makes it difficult to handle.

You can let the test package structure as it is; you just put the actual test modules into a tests subpackage, i.e. instead of

test_subpackage test_module ...

you have

test_subpackage tests test_module ...