search

saghul / pycares

Python interface for c-ares
https://pypi.org/project/pycares/
MIT License
162 stars 74 forks source link

4.3.0: pytest is failing in `tests/test_all.py::DNSTest::test_custom_resolvconf` unit #186

Open kloczek opened 1 year ago

kloczek commented 1 year ago

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

Module si build with PYCARES_USE_SYSTEM_LIB=1. Here is pytest output:

```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pycares-4.3.0-3.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pycares-4.3.0-3.fc35.x86_64/usr/lib/python3.8/site-packages + /usr/bin/pytest -ra -m 'not network' ============================= test session starts ============================== platform linux -- Python 3.8.16, pytest-7.2.2, pluggy-1.0.0 rootdir: /home/tkloczko/rpmbuild/BUILD/pycares-pycares-4.3.0 collected 54 items tests/test_all.py ......F...........xx.........s.................x..xs.. [100%] =================================== FAILURES =================================== ________________________ DNSTest.test_custom_resolvconf ________________________ self = @unittest.skipIf(sys.platform == 'win32', 'skipped on Windows') def test_custom_resolvconf(self): self.result, self.errorno = None, None def cb(result, errorno): self.result, self.errorno = result, errorno self.channel = pycares.Channel(tries=1, timeout=2.0, resolvconf_path=os.path.join(FIXTURES_PATH, 'badresolv.conf')) self.channel.query('google.com', pycares.QUERY_TYPE_A, cb) self.wait() self.assertEqual(self.result, None) > self.assertEqual(self.errorno, pycares.errno.ARES_ETIMEOUT) E AssertionError: 11 != 12 tests/test_all.py:606: AssertionError =========================== short test summary info ============================ SKIPPED [1] tests/test_all.py:430: ANY type does not work on Mac. SKIPPED [1] tests/test_all.py:537: The site used for this test no longer returns a non-ascii SOA. XFAIL tests/test_all.py::DNSTest::test_getnameinfo_ipv6 - reason: XFAIL tests/test_all.py::DNSTest::test_getnameinfo_ipv6_ll - reason: XFAIL tests/test_all.py::DNSTest::test_query_txt_bytes2 - reason: XFAIL tests/test_all.py::DNSTest::test_query_txt_multiple_chunked_with_non_ascii_content - reason: FAILED tests/test_all.py::DNSTest::test_custom_resolvconf - AssertionError: 1... ============= 1 failed, 47 passed, 2 skipped, 4 xfailed in 11.29s ============== ```

Here is list of installed modules in build env

```console Package Version ----------------------------- ----------------- alabaster 0.7.13 attrs 22.2.0 Babel 2.12.1 build 0.10.0 cffi 1.15.1 charset-normalizer 3.1.0 distro 1.8.0 docutils 0.19 exceptiongroup 1.0.0 gpg 1.18.0-unknown idna 3.4 imagesize 1.4.1 importlib-metadata 6.1.0 iniconfig 2.0.0 Jinja2 3.1.2 libcomps 0.1.19 MarkupSafe 2.1.2 packaging 23.0 pip 23.0.1 pluggy 1.0.0 ply 3.11 pycparser 2.21 Pygments 2.14.0 pyproject_hooks 1.0.0 pytest 7.2.2 python-dateutil 2.8.2 pytz 2023.2 requests 2.28.2 rpm 4.17.0 setuptools 65.6.3 six 1.16.0 snowballstemmer 2.2.0 Sphinx 6.1.3 sphinx-rtd-theme 1.2.0 sphinxcontrib-applehelp 1.0.4 sphinxcontrib-devhelp 1.0.2.dev20230202 sphinxcontrib-htmlhelp 2.0.0 sphinxcontrib.jquery 3.0.0 sphinxcontrib-jsmath 1.0.1.dev20230128 sphinxcontrib-qthelp 1.0.3.dev20230128 sphinxcontrib-serializinghtml 1.1.5 tomli 2.0.1 urllib3 1.26.15 wheel 0.38.4 zipp 3.15.0 ```
saghul commented 1 year ago
  • build is performed in env which is cut off from access to the public network (pytest is executed with -m "not network")

Then a number of the tests will simply fail. The test suite creates real DNS traffic.

kloczek commented 1 year ago
  • build is performed in env which is cut off from access to the public network (pytest is executed with -m "not network")

Then a number of the tests will simply fail. The test suite creates real DNS traffic.

So it would be good to @pytest.mark.network to those units which requires such access. https://docs.pytest.org/en/7.1.x/example/markers.html network mark is already quite widely used in many other python modules test suites.

saghul commented 1 year ago

The test suite does not use pytest, it currently uses the vanilla unittest module.

I'd take a PR improving things here, but as it stands if you are interested in running the tests you need Internet access.

kloczek commented 1 year ago

Correctly written unittest test suite can be handled by pytest. Any units which needs public network should have @pytest.mark.network https://docs.pytest.org/en/7.1.x/example/markers.html as that mark is currently widely used by many other python modules test suites.

saghul commented 1 year ago

PRs are welcome. I currently don't have the time or energy to see if pytest would just work and add decorate functions.

If you'd like to send a PR I'd be happy to review it.