search

zopefoundation / zope.configuration

Extensible system for supporting various kinds of configurations
https://zopeconfiguration.readthedocs.io
Other
1 stars 6 forks source link

4.4.0: pytest is failing #55

Closed kloczek closed 2 years ago

kloczek commented 2 years ago

What I did:

I'm trying to package your module as an rpm package. So I'm using the typical build, install and test cycle used on building packages from non-root account.

Looks like pytest is failing in few units:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-zope-configuration-4.4.0-4.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-zope-configuration-4.4.0-4.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra --import-mode=importlib
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.10.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
Using --randomly-seed=2272070319
rootdir: /home/tkloczko/rpmbuild/BUILD/zope.configuration-4.4.0
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, freezegun-0.4.2, aspectlib-1.5.2, toolbox-0.5, rerunfailures-9.1.1, requests-mock-1.9.3, cov-2.12.1, flaky-3.7.0, benchmark-3.4.1, xdist-2.3.0, pylama-7.7.1, datadir-1.3.1, regressions-2.2.0, xprocess-0.18.1, black-0.3.12, asyncio-0.15.1, subtests-0.5.0, isort-2.0.0, hypothesis-6.14.6, mock-3.6.1, profiling-1.7.0, randomly-3.8.0, nose2pytest-1.0.8, pyfakefs-4.5.1, tornado-0.8.1, twisted-1.13.3, aiohttp-0.3.0, localserver-0.5.0, anyio-3.3.1, trio-0.7.0, Faker-8.12.2, cases-3.6.4, yagot-0.5.0
collected 258 items

src/zope/configuration/tests/test_docutils.py .......                                                                                                                [  2%]
src/zope/configuration/tests/test_name.py .......FF.........                                                                                                         [  9%]
src/zope/configuration/tests/test___init__.py ..                                                                                                                     [ 10%]
src/zope/configuration/tests/test_fields.py .................................                                                                                        [ 23%]
src/zope/configuration/tests/test_zopeconfigure.py ..                                                                                                                [ 24%]
src/zope/configuration/tests/test_docs.py .                                                                                                                          [ 24%]
src/zope/configuration/tests/test_config.py ........................................................................................................................ [ 70%]
..........                                                                                                                                                           [ 74%]
src/zope/configuration/tests/test_xmlconfig.py ....................................................F.F..........                                                     [100%]

================================================================================= FAILURES =================================================================================
______________________________________________________________________ Test_path.test_relative_empty _______________________________________________________________________

self = <test_name.Test_path testMethod=test_relative_empty>

    def test_relative_empty(self):
        import os
>       self.assertEqual(self._callFUT('', 'zope.configuration.tests'),
                         os.path.dirname(__file__))
E       AssertionError: '/hom[20 chars]BUILDROOT/python-zope-configuration-4.4.0-4.fc[62 chars]ests' != '/hom[20 chars]BUILD/zope.configuration-4.4.0/src/zope/configuration/tests'
E       - /home/tkloczko/rpmbuild/BUILDROOT/python-zope-configuration-4.4.0-4.fc35.x86_64/usr/lib/python3.8/site-packages/zope/configuration/tests
E       ?                              ---- -------    ^                   -------------- -  ---------------------- -----
E       + /home/tkloczko/rpmbuild/BUILD/zope.configuration-4.4.0/src/zope/configuration/tests
E       ?                                   ^

src/zope/configuration/tests/test_name.py:121: AssertionError
______________________________________________________________________ Test_path.test_relative_w_file ______________________________________________________________________

self = <test_name.Test_path testMethod=test_relative_w_file>

    def test_relative_w_file(self):
        import os
>       self.assertEqual(
            self._callFUT('configure.zcml', 'zope.configuration.tests'),
            os.path.join(os.path.dirname(__file__), 'configure.zcml'))
E       AssertionError: '/hom[20 chars]BUILDROOT/python-zope-configuration-4.4.0-4.fc[77 chars]zcml' != '/hom[20 chars]BUILD/zope.configuration-4.4.0/src/zope/config[24 chars]zcml'
E       - /home/tkloczko/rpmbuild/BUILDROOT/python-zope-configuration-4.4.0-4.fc35.x86_64/usr/lib/python3.8/site-packages/zope/configuration/tests/configure.zcml
E       ?                              ---- -------    ^                   -------------- -  ---------------------- -----
E       + /home/tkloczko/rpmbuild/BUILD/zope.configuration-4.4.0/src/zope/configuration/tests/configure.zcml
E       ?                                   ^

src/zope/configuration/tests/test_name.py:126: AssertionError
___________________________________________________________________ Test_file.test_wo_execute_w_context ____________________________________________________________________

self = <test_xmlconfig.Test_file testMethod=test_wo_execute_w_context>

    def test_wo_execute_w_context(self):
        from zope.configuration import xmlconfig
        from zope.configuration.config import ConfigurationMachine
        from zope.configuration.xmlconfig import registerCommonDirectives
        from zope.configuration.tests import samplepackage
        from zope.configuration.tests.samplepackage import foo
        context = ConfigurationMachine()
        context.package = samplepackage
        registerCommonDirectives(context)
        file_name = path("samplepackage", "configure.zcml")
        logger = LoggerStub()
        with _Monkey(xmlconfig, logger=logger):
            ret = self._callFUT('configure.zcml', context=context,
                                execute=False)
        self.assertTrue(ret is context)
        self.assertEqual(len(logger.debugs), 1)
>       self.assertEqual(logger.debugs[0], ('include %s', (file_name,), {}))
E       AssertionError: Tuples differ: ('inc[36 chars]BUILDROOT/python-zope-configuration-4.4.0-4.fc[98 chars], {}) != ('inc[36 chars]BUILD/zope.configuration-4.4.0/src/zope/config[45 chars], {})
E
E       First differing element 1:
E       ('/ho[21 chars]BUILDROOT/python-zope-configuration-4.4.0-4.fc[93 chars]ml',)
E       ('/ho[21 chars]BUILD/zope.configuration-4.4.0/src/zope/config[40 chars]ml',)
E
E         ('include %s',
E       -  ('/home/tkloczko/rpmbuild/BUILDROOT/python-zope-configuration-4.4.0-4.fc35.x86_64/usr/lib/python3.8/site-packages/zope/configuration/tests/samplepackage/configure.zcml',),
E       ?                                 ---- -------    ^                   -------------- -  ---------------------- -----
E
E       +  ('/home/tkloczko/rpmbuild/BUILD/zope.configuration-4.4.0/src/zope/configuration/tests/samplepackage/configure.zcml',),
E       ?                                      ^
E
E          {})

src/zope/configuration/tests/test_xmlconfig.py:757: AssertionError
______________________________________________________________ Test_file.test_wo_execute_wo_context_w_package ______________________________________________________________

self = <test_xmlconfig.Test_file testMethod=test_wo_execute_wo_context_w_package>

    def test_wo_execute_wo_context_w_package(self):
        from zope.configuration import xmlconfig
        from zope.configuration.tests import samplepackage
        from zope.configuration.tests.samplepackage import foo
        file_name = path("samplepackage", "configure.zcml")
        logger = LoggerStub()
        with _Monkey(xmlconfig, logger=logger):
            context = self._callFUT('configure.zcml', package=samplepackage,
                                    execute=False)
        self.assertEqual(len(logger.debugs), 1)
>       self.assertEqual(logger.debugs[0], ('include %s', (file_name,), {}))
E       AssertionError: Tuples differ: ('inc[36 chars]BUILDROOT/python-zope-configuration-4.4.0-4.fc[98 chars], {}) != ('inc[36 chars]BUILD/zope.configuration-4.4.0/src/zope/config[45 chars], {})
E
E       First differing element 1:
E       ('/ho[21 chars]BUILDROOT/python-zope-configuration-4.4.0-4.fc[93 chars]ml',)
E       ('/ho[21 chars]BUILD/zope.configuration-4.4.0/src/zope/config[40 chars]ml',)
E
E         ('include %s',
E       -  ('/home/tkloczko/rpmbuild/BUILDROOT/python-zope-configuration-4.4.0-4.fc35.x86_64/usr/lib/python3.8/site-packages/zope/configuration/tests/samplepackage/configure.zcml',),
E       ?                                 ---- -------    ^                   -------------- -  ---------------------- -----
E
E       +  ('/home/tkloczko/rpmbuild/BUILD/zope.configuration-4.4.0/src/zope/configuration/tests/samplepackage/configure.zcml',),
E       ?                                      ^
E
E          {})

src/zope/configuration/tests/test_xmlconfig.py:732: AssertionError
========================================================================= short test summary info ==========================================================================
FAILED src/zope/configuration/tests/test_name.py::Test_path::test_relative_empty - AssertionError: '/hom[20 chars]BUILDROOT/python-zope-configuration-4.4.0-4.fc[62 chars...
FAILED src/zope/configuration/tests/test_name.py::Test_path::test_relative_w_file - AssertionError: '/hom[20 chars]BUILDROOT/python-zope-configuration-4.4.0-4.fc[77 char...
FAILED src/zope/configuration/tests/test_xmlconfig.py::Test_file::test_wo_execute_w_context - AssertionError: Tuples differ: ('inc[36 chars]BUILDROOT/python-zope-configu...
FAILED src/zope/configuration/tests/test_xmlconfig.py::Test_file::test_wo_execute_wo_context_w_package - AssertionError: Tuples differ: ('inc[36 chars]BUILDROOT/python-z...
====================================================================== 4 failed, 254 passed in 1.39s =======================================================================
pytest-xprocess reminder::Be sure to terminate the started process by running 'pytest --xkill' if you have not explicitly done so in your fixture with 'xprocess.getinfo(<process_name>).terminate()'.

What I expect to happen:

pytest should finish without errors and fails.

What version of Python and Zope/Addons I am using:

Python 3.8.12

d-maurer commented 2 years ago

Tomasz Kłoczko wrote at 2021-9-22 14:45 -0700:

I'm trying to package your module as an rpm package. So I'm using the typical build, install and test cycle used on building packages from non-root account.

  • "setup.py build"
  • "setup.py install --root </install/prefix>"
  • "pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

When will you learn that most zopefoundation packages have been designed for zope.testrunner (not pytest) as test runner?

Usually, the tests are activated via tox, hiding test runner details and testing various test variants (e.g. with different Python versions).

It is unlikely that someone will take the effort to support more than a single test runner even if you report serveral pytest failures every day.

kloczek commented 2 years ago

When will you learn that most zopefoundation packages have been designed for zope.testrunner (not pytest) as test runner?

Issue is that this is no longer true. At the moment MAJORYTY of all zope modules is possible to test using pytest and only few of them reports only minor issues.

d-maurer commented 2 years ago

Tomasz Kłoczko wrote at 2021-9-22 16:46 -0700:

When will you learn that most zopefoundation packages have been designed for zope.testrunner (not pytest) as test runner?

Issue is that this is no longer true. At the moment MAJORYTY of all zope modules is possible to test using pytest and only few of them reports only minor issues.

I wrote "most ... have been DESIGNED FOR ...". This does not exclude that pytext may work BY ACCIDENT.

A few packages have tests designed for pytest. Thus, start tests via tox; the tox configuration handles details regarding the test runner. It also installs required test dependencies.