I'm trying to package your module as an rpm package. So I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix>
• run pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

Here is pytest output:

Is there any reason you're not using the testtools test runner?

I have 11 reasons 😄 😋

[tkloczko@devel-g2v SPECS]$ grep  'python3dist(testtools)' *
python-fasteners.spec:BuildRequires:    python3dist(testtools)
python-fixtures.spec:BuildRequires:     python3dist(testtools)  >= 0.9.22
python-hacking.spec:BuildRequires:      python3dist(testtools)
python-requests-mock.spec:BuildRequires:        python3dist(testtools)
python-subunit.spec:BuildRequires:      python3dist(testtools)  >= 2.5
python-testrepository.spec:BuildRequires:       python3dist(testtools)
python-testresources.spec:BuildRequires:        python3dist(testtools)
python-testscenarios.spec:BuildRequires:        python3dist(testtools)
python-traceback2.spec:BuildRequires:   python3dist(testtools)
python-zope-testrunner.spec:BuildRequires:      python3dist(testtools)
subunit.spec:BuildRequires:     python3dist(testtools)  >= 1.8.0

I don't understand your reasons, nor why you are insisting on using an entirely different testing framework to test this package.

pytest is not equal to unittest, and unittest is not equal to testtools, which is not equal to nosetests and so on throughout the ecosystem.

PEP-517 doesn't mandate py.test at all, and bugs with PEP-517 support would be taken seriously.

OTOH I'm largely emeritus here, so if you convince folk that py.test compatibility should be worked on, so be it - but it makes no sense on the face of it.

I don't understand your reasons, nor why you are insisting on using an entirely different testing framework to test this package.

Because in my packaging approach if test suite is provided and it is able to test something it needs to be used. More test -> more opportunities to catch some issue not only in tested code but whole tooling used by exact package test suite as well -> quality of whole packages distribution is going up.

pytest is not equal to unittest, and unittest is not equal to testtools, which is not equal to nosetests and so on throughout the ecosystem.

Enough fresh unittest test suite can be correctly handled by pytest. Some stat about how often I was able to use so far pytest in case of +900 modules population

[tkloczko@devel-g2v SPECS]$ grep ^%pytest python-* | wc -l; grep ^%tox python-* | wc -l; grep ^%py3_test python-*|wc -l; ls -1 python-* | wc -l
897
1
7
940

As you see almost 95% of all packages modules I was able to use pytest. In last two days I was able to increase tag stat by 5% using patched pytest using https://github.com/pytest-dev/pytest/issues/9976. My target is to have 100%.

So, entirely in favour of more testing. Thats great. But its entirely irrelevant to this discussion.

If py.test want to offer support for testtools extended protocols, thats up to py.test to offer. testtools only guarantees that it can be tested with itself.

For the starter please try to have look on DeprecationWarning warnings.

Please try to understand me .. I'm just a messenger. As you see I have my own goals in which using pytest everywhere has its crucial value. Whole "painted by pytest picture" does not need to be perfect. At least needs to show me "a picture" which I'm going to use as baseline for regression tests in the future. In that kind of approach any changes on that "picture" during day-to-day whole distribution updates would be early signal that change in package A affects packages X, Y, Z. That signal will be generated almost instantly. As you can see I can deliver you in the future completely automatically generated report with exact impact on some population of other modules which are using your module. If you may help me now bringing pytest to kind of useable form I can help you lowering the risk releasing something which may trash/not work other people projects. That is my kind of offer .. Are you interested in such an offer? 😋

I don't think you're going to get any traction trying to test with pytest here. Testtools can test itself, and it meets your needs to test like that as far as I can tell, so I don't understand the resistance to using it. As Robert says, it looks like pytest lacks testtools extended protocols.

gentle ping .. just tested and looks like 2.6.0 still fails in the same units 🤔

If you want this to happen, you're very likely to need to figure out how to get it working for yourself and send PRs. Pings probably won't get anyone excited to work on testing one test-running framework using a different test-running framework.

Hmm .. so you've not been able to reproduce those fails? 🤔

Since it's not our supported way to run the test suite, I haven't spent any time even trying.

Since it's not our supported way to run the test suite, I haven't spent any time even trying.

pytest shows many warnings which is not possible to see with your way of testing. At least please try to review those warnings. As well pytest is de facto standard test suite framework with many extensions. Using it really can make maintenance of your module easier. Errors shows as well shows code issues.

Other issue is that tet suite (testtools/tests/) is part of the installed resources. This part should be not included in .whl archive.

And update of latest version of pytest ourput (now it fails one unit more)

If you want this sort of thing improved, you're going to have to put the work in and send patches yourself.

And you will changes related to pytest? 🤔

(I assume you meant "you will review".)

Reviewing PRs is at least a lot more likely that being receptive to incessant harassment via issues.

Just tested 2.6.0 and looks like now pytest is failing in 40 units

Just FTR 2.7.0 pytest output