search

pytest-dev / pytest

The pytest framework makes it easy to write small tests, yet scales to support complex functional testing
https://pytest.org
MIT License
11.64k stars 2.59k forks source link

Bring back link checks #12474

Open The-Compiler opened 2 weeks ago

The-Compiler commented 2 weeks ago

We have quite some history when it comes to link checks:

So we're currently not running link checking at all. And it shows:

.../adopt.rst:56: WARNING: broken link: fixture.html ()
.../adopt.rst:55: WARNING: broken link: assert.html ()
.../talks.rst:88: WARNING: broken link: http://bruynooghe.blogspot.com/2010/06/pytest-test-generators-and-cached-setup.html (404 Client Error: Not Found for url: http://bruynooghe.blogspot.com/2010/06/pytest-test-generators-and-cached-setup.html)
.../talks.rst:103: WARNING: broken link: http://bruynooghe.blogspot.com/2009/12/skipping-slow-test-by-default-in-pytest.html (404 Client Error: Not Found for url: http://bruynooghe.blogspot.com/2009/12/skipping-slow-test-by-default-in-pytest.html)
.../announce/release-2.2.0.rst:12: WARNING: broken link: http://pytest.org/en/stable/example/how-to/parametrize.html (404 Client Error: Not Found for url: https://pytest.org/en/stable/example/how-to/parametrize.html)
.../adopt.rst:55: WARNING: broken link: https://bitbucket.org/hpk42/pycmd/overview (404 Client Error: Not Found for url: https://bitbucket.org/hpk42/pycmd/overview)
.../announce/release-2.6.0.rst:21: WARNING: broken link: https://bitbucket.org/pytest-dev/pytest/issues?status=new&status=open&sort=-votes (404 Client Error: Not Found for url: https://bitbucket.org/pytest-dev/pytest/issues?status=new&status=open&sort=-votes)
.../history.rst:120: WARNING: broken link: http://web.archive.org/web/20091012022829/http://codespeak.net/py/dist/test/plugin/how-to/monkeypatch.html (404 Client Error: NOT FOUND for url: http://web.archive.org/web/20091012022829/http://codespeak.net/py/dist/test/plugin/how-to/monkeypatch.html)
.../explanation/goodpractices.rst:113: WARNING: broken link: https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure> (Anchor 'the-structure>' not found)
.../announce/release-2.3.0.rst:97: WARNING: broken link: https://docs.pytest.org/en/6.0.1/faq.html (404 Client Error: Not Found for url: https://docs.pytest.org/en/6.0.1/faq.html)
.../announce/release-8.0.0rc2.rst:22: WARNING: broken link: https://docs.pytest.org/en/release-8.0.0rc2/changelog.html (404 Client Error: Not Found for url: https://docs.pytest.org/en/release-8.0.0rc2/changelog.html)
.../reference/reference.rst:153: WARNING: broken link: https://github.com/asottile/re-assert#re_assertmatchespattern-str-args-kwargs (Anchor 're_assertmatchespattern-str-args-kwargs' not found)
.../contact.rst:24: WARNING: broken link: https://matrix.to/#/%23pytest:libera.chat (Anchor '/%23pytest:libera.chat' not found)
.../changelog.rst:5757: WARNING: broken link: https://peps.python.org/pep-0506/#recommended-filter-settings-for-test-runners (Anchor 'recommended-filter-settings-for-test-runners' not found)
.../history.rst:99: WARNING: broken link: https://pypi.org/project/py/0.8.0-alpha2/#history/ (Anchor 'history/' not found)
.../contributing.rst:150: WARNING: broken link: https://tox.readthedocs.io/en/latest/config.html#configuration-discovery (Anchor 'configuration-discovery' not found)
.../contact.rst:24: WARNING: broken link: https://web.libera.chat/#pytest (Anchor 'pytest' not found)
.../adopt.rst:57: WARNING: broken link: mark.html ()
.../talks.rst:83: WARNING: broken link: monkeypatch.html ()
.../talks.rst:87: WARNING: broken link: parametrize.html#test-generators ()
.../talks.rst:105: WARNING: broken link: plugins.html ()
.../adopt.rst:47: WARNING: broken link: unittest.html ()
.../changelog.rst:1157: WARNING: broken link: with-setup-nose ()
.../adopt.rst:59: WARNING: broken link: xdist.html ()
.../adopt.rst:56: WARNING: broken link: xunit_setup.html ()
.../explanation/goodpractices.rst:290: WARNING: broken link: https://github.com/pytest-dev/pytest-runner#deprecation-notice (Anchor 'deprecation-notice' not found)
.../history.rst:108: WARNING: broken link: https://github.com/pytest-dev/pytest/blob/main/doc/en/changelog.rst#091 (Anchor '091' not found)
.../changelog.rst:104: WARNING: broken link: https://github.com/python/cpython/blob/51aefc5bf907ddffaaf083ded0de773adcdf08c8/Lib/unittest/case.py#L419-L426 (Anchor 'L419-L426' not found)
.../changelog.rst:6716: WARNING: broken link: https://github.com/thisch (404 Client Error: Not Found for url: https://github.com/thisch)

We should bring it back somehow. According to @webknjaz it got better at doing retries and such, and apparently there's also a distinct status for timeouts:

From @nicoddemus in #6916:

The idea is good, but I'm not sure if the scheduled based job won't just be ignored in the end.

But what we currently have is worse IMHO...

nicoddemus commented 2 weeks ago

The idea is good, but I'm not sure if the scheduled based job won't just be ignored in the end.

I reconsidered this, because now we have some scheduled jobs (like plugin update, pre-commit, etc) and history shows that they are not ignored, often sometimes takes time to fix them if they fail.

So a scheduled job might do the trick here in the end.