Sphinx linkcheck and broken/redirect occurrences in Python Docs

[rffontenelle]

Running make linkcheck in Doc folder outputs thousands of broken and redirected status. It would be nice to clean this up so we can link-check Python in CI/CD for each commit, but right now it is too polluted.

Some of these occurrences could/should be fixed in the docs itself, others can benefit from Sphinx's linkcheck configs e.g. linkcheck_ignore and linkcheck_allow_redirect.

See linkcheck-log.txt for the full log as of commit f2b7ecb.

Fun stats obtained from the above file:

• Of its 8327 lines, where 7824 are related to BPO -> GH issues. Of 7824, 5744 lines are from whatsnew/changelog and only 20 are not from whatsnew/
• 241 lines are redirection of CPython CVS URL, fixing /tree/ to /blob/ in GitHub URL
• 28 lines are broken links, where one broken link is 'https://' from an example in whatsnet/changelog

The way I see, this steps divide in:

• Clean BPO to GH redirection messages
• Fix broken links
• Clean CPython CVS URL redirections
• Clean GH issues to GH PR redirections
• Fix the rest of the occurrences.

linkcheck_allow_redirect and linkcheck_ignore can be very handy in this case. linkcheck_allow_redirect makes 'ok' status a redirect that is being spotted by linkcheck, and we have linkcheck_ignore as the last resource.

Questions I have before implementing the solution:

• Documentation hosted by Read The Docs may have language enabled so example.com is redirected to example.com/en/latest. To handle occurrences, I could add them to linkcheck_allow_redirect or we can use sphinx-ext-intersphinx to map a keyword to the documentation URL (e.g 'rtd' for read-the-docs docs). The last option allows to map to proper language of the target URL linked, similar on how Weblate did)
• Is there any restrictions to fix broken/redirect links in old whatsnew/.rst?
• Is there any restrictions to fix broken/redirect links in old whatsnew/changelog.rst (i.e. Misc/NEWS)?
• Should I create a single Pull Request for all the fixes?

Linked PRs

• gh-103569
• gh-103602
• gh-103608
• gh-103683
• gh-104001
• gh-104088
• gh-123019

[merwok]

What about ignoring the 7824 bpo redirects? They work fine, we control bpo and can keep the redirector working, we don’t need to change 8000 lines in historical whatsnew docs.

The CVS redirections can be fixed, as well as the broken links.

[hugovk]

(For reference, some earlier discussion at https://discuss.python.org/t/sphinx-linkcheck-and-broken-redirect-occurrences-in-python-docs/25687)

• Of its 8327 lines, where 7824 are related to BPO -> GH issues. Of 7824, 5744 lines are from whatsnew/changelog and only 20 are not from whatsnew/

Yes, let's ignore the BPO redirects. There's so many, I think either linkcheck_allow_redirect or linkcheck_ignore, whichever is quicker for linkcheck to process.

We actually have a linkcheck_ignore for BPO, but the regex doesn't match, I guess it's gone out of sync:

https://github.com/python/cpython/blob/7f3c10650385907b5a2234edb2b1334cafd47a0a/Doc/conf.py#L258

• 241 lines are redirection of CPython CVS URL, fixing /tree/ to /blob/ in GitHub URL

Please could you give some examples of these?

• Documentation hosted by Read The Docs may have language enabled so example.com is redirected to example.com/en/latest. To handle occurrences, I could add them to linkcheck_allow_redirect or we can use sphinx-ext-intersphinx to map a keyword to the documentation URL (e.g 'rtd' for read-the-docs docs). The last option allows to map to proper language of the target URL linked, similar on how Weblate did)

If there's not too many, we could add to linkcheck_allow_redirect, similar to the devguide. There may be some other useful rules to copy over:

https://github.com/python/devguide/blob/79e1a1f2961ba0882d5f0274f7c8876e36ca5dff/conf.py#L60-L107

• Is there any restrictions to fix broken/redirect links in old whatsnew/.rst?
• Is there any restrictions to fix broken/redirect links in old whatsnew/changelog.rst (i.e. Misc/NEWS)?

Not as such, although effort is usually prioritised for the newer/more active pages. Cleaning them up would mean linkcheck output is more useful in the future.

The way I see, this steps divide in:

• Clean BPO to GH redirection messages
• Fix broken links
• Clean CPython CVS URL redirections
• Clean GH issues to GH PR redirections
• Fix the rest of the occurrences.

...

• Should I create a single Pull Request for all the fixes?

Maybe a PR for each of these steps? And split smaller if you feel they're too big. Smaller PRs are easier to review and get merged.

[rffontenelle]

• 241 lines are redirection of CPython CVS URL, fixing /tree/ to /blob/ in GitHub URL

Please could you give some examples of these?

Sure. The file Doc/about.rst has :source:`Misc/ACKS`, which links to that file in CPython's source code repository. The custom role "source" is defined in Doc/tools/extensions/pyspecific.py to https://github.com/python/cpython/tree/main/ + the suffix.

GitHub URL for browsing files is /blob/ and directories is /tree/. When browsing Misc/ACKS, the /tree/ redirects to /blob/.

From linkcheck stdout:

about: line   33) redirect  https://github.com/python/cpython/tree/main/Misc/ACKS - permanently to https://github.com/python/cpython/blob/main/Misc/ACKS

See in the above log how the redirect was from /tree/ to /blob/.

• Is there any restrictions to fix broken/redirect links in old whatsnew/.rst?
• Is there any restrictions to fix broken/redirect links in old whatsnew/changelog.rst (i.e. Misc/NEWS)?

Not as such, although effort is usually prioritised for the newer/more active pages. Cleaning them up would mean linkcheck output is more useful in the future.

Great, I'll focus on the cleanup then.

• Should I create a single Pull Request for all the fixes?

Maybe a PR for each of these steps? And split smaller if you feel they're too big. Smaller PRs are easier to review and get merged.

Will do, thanks for the suggestion.

[hugovk]

Thanks, we can add the tree->blob redirects to linkcheck_allow_redirect like: https://github.com/python/devguide/blob/main/conf.py#L73-L74

[merwok]

Why not do the one-line fix to pyspecific rather?

[rffontenelle]

Because in :source:`something` the URL has /tree/ if something is a directory and /blob/ if it is a file. GitHub redirects both ways. If we put /blob/ in pyspecific.py, then a :source:`Misc/` would result in /blob/ being redirected to /tree/.

EDIT: See some examples:

$ curl -sI https://github.com/python/cpython/blob/main/Misc | grep -E '(^HTTP|^location:)'
HTTP/2 301 
location: https://github.com/python/cpython/tree/main/Misc
$ curl -sI https://github.com/python/cpython/tree/main/Misc/ACKS | grep -E '(^HTTP|^location:)'
HTTP/2 301 
location: https://github.com/python/cpython/blob/main/Misc/ACKS

[rffontenelle]

Suggestion please?

bpo-31453: Add TLSVersion constants and SSLContext.maximum_version / minimum_version attributes. The new API wraps OpenSSL 1.1 https://www.openssl.org/docs/man1.1.0/ssl/SSL_CTX_set_min_proto_version.html feature.

The text talks about OpenSSL 1.1.0 and the URL for version 1.1.0 is broken (HTTP 404), but there are valid URLs for the same resource pointing to OpenSSL version 1.1.1 and to 3.1.0. Should I update the URL to either one, or use wayback machine to make sure to match the OpenSSL version?

[merwok]

If openssl 1.1.1 is a bugfix release of the 1.1 branch, then it would be fine to point there.

[rffontenelle]

Created the first PR. I'm planning to create other 2 PRs, with more file changes than this one.

[rffontenelle]

I noticed the docs.yml workflow file runs for all branches from main to 3.7. Should I back-port the fixes to all these versions (cases were the bot won't be able to back-port automatically) or is there some code freeze that prevents it?

[merwok]

Backporting doc fixes to 3.11 is fine, but older branches only accept security fixes.

[rffontenelle]

Got it. It means a linkcheck job in docs.yml will require a condition check to exclude versions < 3.11. Thanks.

[merwok]

A workflow in the main branch applies to other branches?! What a strange system.

[rffontenelle]

Yep. It is a centralized way to control what routines will run in which events (push, pull requests, etc.) and in which branches.

[hugovk]

Are you sure?

I made a test repo with main and 3.11 branches:

• https://github.com/hugovk/test-workflows

Each branch has a workflow that does echo "hi from <branch-name>":

• https://github.com/hugovk/test-workflows/blob/main/.github/workflows/doc.yml
• https://github.com/hugovk/test-workflows/blob/3.11/.github/workflows/doc.yml

A push to each branch echoes the matching version number. So it runs its own workflow, not the one from main:

• https://github.com/hugovk/test-workflows/actions/runs/4720278815/jobs/8372127403
• https://github.com/hugovk/test-workflows/actions/runs/4720286253/jobs/8372144096

Also, making branches from each, and a PR back, also echoes the matching version number, not the one from main:

• https://github.com/hugovk/test-workflows/pull/1
  • https://github.com/hugovk/test-workflows/actions/runs/4720297470/jobs/8372169004?pr=1
• https://github.com/hugovk/test-workflows/pull/2
  • https://github.com/hugovk/test-workflows/actions/runs/4720299864/jobs/8372174031?pr=2

[rffontenelle]

I was sure until now. I mean, what's the point of having branch filters with they don't actually filter on which branch a workflow can or cannot be run?

on:
  push:
    branches:
      - main
      - '3.11'
      - '3.10'
    ...

[hugovk]

I expect it's to have as few changes as possible between branches to make it easier for the bot to backport (and for humans when the bot can't do it).

[rffontenelle]

Well, I stand corrected. That means a linkcheck job will need to be added to both main and 3.11. No problem.

[hugovk]

We probably don't want to run linkcheck on the CI for a couple of reasons:

• it takes about 50 minutes to run
• we can get new failures when things change external to a PR, like a website going down or temporary network failures

[rffontenelle]

Fixes to broken links now submitted.

[rffontenelle]

We probably don't want to run linkcheck on the CI for a couple of reasons:

• it takes about 50 minutes to run

To reduce this time, a workaround is to include in linkcheck_ignore the links to https://bugs.python.org/ and https://github.com/python/cpython/issues/. A local test (editing conf.py) reduced the time to just 5 minutes!

I would prefer this is happen only via command-line option e.g. make -C Doc linkcheck SPHINXOPTS='--keep-going -D linkcheck_ignore=<expr>', but I was unable to find the proper <expr>. Any suggestion?

• we can get new failures when things change external to a PR, like a website going down or temporary network failures

Indeed, this is something out of control of cpython. We could set higher value in linkcheck_retries and linkcheck_timeout, though.

[merwok]

I would prefer this is happen only via command-line option

What about defining a custom environment variable (instead of using SPHINXOPTS) that you can read in the config code?

[rffontenelle]

What about defining a custom environment variable (instead of using SPHINXOPTS) that you can read in the config code?

OMG, awesome idea.

It is simply the case of checking if $CI environment variable is set, because GitHub Actions sets it. Then, insert the desired URLs:

if 'CI' in os.environ:
    linkcheck_ignore.insert(0, r'https://bugs.python.org/.*')
    linkcheck_ignore.insert(1, r'https://github.com/python/cpython/issues/\d+') 

It does reduce run time to 5 minutes. See my test.

[rffontenelle]

@hugovk have you considered my idea on adding linkcheck to GitHub Actions with 1) linkcheck_ignore BPO and GH issues to reduce time to 5 minutes; 2) set higher timeout and retries values ?

Also, I recommend disabling -W flag to error on broken links, but setting up a problem matcher to annotate both broken (as error) and redirects (as warning).

I can propose a PR for better analysis.

[rffontenelle]

New PR, now aiming occurrences of "redirected permanently". After this one, there is only "redirected with Found" and linkcheck is OK.

[rffontenelle]

After some time, I see new wild links popping up from linkcheck output.

My main problems now are these, which I hope to get :

1. pypi role via extlinks, PyPI's URL normalization and "redirected permanently": conf.py uses extlinks extension to create the :pypi:`packagename` role, but PyPI is normalizing the URL to lowercase letters and dash instead of underscore. So we're having "redirected permanently" status on these links. For instance, :pypi:`Urwid` results in https://pypi.org/project/Urwid/, but PyPI redirects to https://pypi.org/project/urwid/.
   Is it possible for extlinks to generate an URL with the proper replacements (lowercase letters and dash instead of underscore) so I can avoid adding an ignore exception to linkcheck?

2. Links to other Sphinx-based documentation causing "redirect with found" Python docs links to many other Sphinx-based documentation, which have its URL appended with language and version (e.g. https://babel.pocoo.org/ to https://babel.pocoo.org/en/latest/). Adding exceptions or Interpshinx might add a lot of lines to conf.py, because it would be a least one line per documentation URL

For the record, these are the currently non-ok output of linkcheck:

status "broken"

howto/curses.rst:541: [broken] https://linux.die.net/man/3/ncurses: 403 Client Error: Forbidden for url: https://linux.die.net/man/3/ncurses
library/functools.rst:220: [broken] https://en.wikipedia.org/wiki/Cache_replacement_policies#Least_recently_used_(LRU): Anchor 'Least_recently_used_%28LRU%29' not found
library/select.rst:279: [broken] https://linux.die.net/man/4/epoll: 403 Client Error: Forbidden for url: https://linux.die.net/man/4/epoll
library/ssl.rst:2757: [broken] https://www.openssl.org/docs/man1.1.1/man1/ciphers.html#CIPHER-LIST-FORMAT: Anchor 'CIPHER-LIST-FORMAT' not found
library/statistics.rst:995: [broken] https://www.statisticshowto.com/probability-and-statistics/z-score/: 403 Client Error: Forbidden for url: https://www.statisticshowto.com/probability-and-statistics/z-score/
whatsnew/2.4.rst:759: [broken] https://developer-old.gnome.org/glib/2.26/: HTTPSConnectionPool(host='developer-old.gnome.org', port=443): Max retries exceeded with url: /glib/2.26/ (Caused by NameResolutionError(": Failed to resolve 'developer-old.gnome.org' ([Errno -2] Name or service not known)"))
whatsnew/2.6.rst:191: [broken] https://svn.python.org/view/tracker/importer/: unauthorized
whatsnew/3.12.rst:361: [broken] https://github.com/python/cpython/tree/main/Modules/_xxsubinterpretersmodule.c: 404 Client Error: Not Found for url: https://github.com/python/cpython/tree/main/Modules/_xxsubinterpretersmodule.c
whatsnew/3.2.rst:1650: [broken] https://www.openssl.org/docs/man1.0.2/man1/ciphers.html#CIPHER-LIST-FORMAT: Anchor 'CIPHER-LIST-FORMAT' not found
whatsnew/3.4.rst:1962: [broken] https://ltp.sourceforge.net/coverage/lcov.php: 404 Client Error: Not Found for url: https://ltp.sourceforge.net/coverage/lcov.php
whatsnew/3.8.rst:2228: [broken] https://ark.intel.com/content/www/us/en/ark/products/76088/intel-core-i7-4960hq-processor-6m-cache-up-to-3-80-ghz.html: 403 Client Error: Forbidden for url: https://ark.intel.com/content/www/us/en/ark/products/76088/intel-core-i7-4960hq-processor-6m-cache-up-to-3-80-ghz.html

status "redirected permanently"

faq/design.rst:330: [redirected permanently] https://www.nuitka.net/ to https://nuitka.net/
faq/design.rst:347: [redirected permanently] https://www.pypy.org to https://pypy.org/
howto/curses.rst:38: [redirected permanently] https://pypi.org/project/Urwid/ to https://pypi.org/project/urwid/
library/datetime.rst:40: [redirected permanently] https://pypi.org/project/DateType/ to https://pypi.org/project/datetype/
library/hashlib.rst:23: [redirected permanently] https://csrc.nist.gov/publications/detail/fips/180/4/final to https://csrc.nist.gov/pubs/fips/180-4/upd1/final
library/hashlib.rst:23: [redirected permanently] https://csrc.nist.gov/publications/detail/fips/202/final to https://csrc.nist.gov/pubs/fips/202/final
library/hashlib.rst:657: [redirected permanently] https://csrc.nist.gov/publications/detail/sp/800-106/archive/2009-02-25 to https://csrc.nist.gov/pubs/sp/800/106/final
library/http.cookiejar.rst:140: [redirected permanently] http://kristol.org/cookie/errata.html to https://kristol.org/cookie/errata.html
library/importlib.metadata.rst:182: [redirected permanently] https://pypi.org/project/backports.entry_points_selectable/ to https://pypi.org/project/backports.entry-points-selectable/
library/json.rst:14: [redirected permanently] https://www.ecma-international.org/publications-and-standards/standards/ecma-404/ to https://ecma-international.org/publications-and-standards/standards/ecma-404/
library/os.rst:4513: [redirected permanently] https://discuss.python.org/t/33555 to https://discuss.python.org/t/concerns-regarding-deprecation-of-fork-with-alive-threads/33555
library/ssl.rst:1566: [redirected permanently] https://www.openssl.org/docs/manmaster/man3/SSL_CTX_load_verify_locations.html to https://docs.openssl.org/master/man3/SSL_CTX_load_verify_locations/
library/ssl.rst:1642: [redirected permanently] https://www.openssl.org/docs/manmaster/man1/ciphers.html to https://docs.openssl.org/master/man1/ciphers/
library/ssl.rst:1876: [redirected permanently] https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_sess_number.html to https://docs.openssl.org/1.1.1/man3/SSL_CTX_sess_number/
library/ssl.rst:2019: [redirected permanently] https://www.openssl.org/docs/manmaster/man3/SSL_CTX_get_security_level.html to https://docs.openssl.org/master/man3/SSL_CTX_get_security_level/
library/tkinter.rst:61: [redirected permanently] https://www.packtpub.com/product/python-gui-programming-with-tkinter/9781788835886 to https://www.packtpub.com/en-us/product/python-gui-programming-with-tkinter-9781788835886
library/typing.rst:2711: [redirected permanently] https://typing.readthedocs.io/en/latest/source/unreachable.html to https://typing.readthedocs.io/en/latest/guides/unreachable.html
library/typing.rst:42: [redirected permanently] https://pypi.org/project/typing_extensions/ to https://pypi.org/project/typing-extensions/
library/xmlrpc.client.rst:168: [redirected permanently] https://xmlrpc-c.sourceforge.net/introspection.html to https://xmlrpc-c.sourceforge.io/introspection.html
reference/introduction.rst:72: [redirected permanently] https://www.pypy.org/ to https://pypy.org/
using/mac.rst:157: [redirected permanently] https://qt.io to https://www.qt.io/
using/windows.rst:1288: [redirected permanently] https://pypi.org/project/PyWin32/ to https://pypi.org/project/pywin32/
using/windows.rst:1308: [redirected permanently] http://timgolden.me.uk/python/win32_how_do_i.html to https://timgolden.me.uk/python/win32_how_do_i.html
using/windows.rst:611: [redirected permanently] https://www.enthought.com/edm/ to https://assets.enthought.com/downloads/edm/
whatsnew/2.4.rst:687: [redirected permanently] http://www.lahey.com/float.htm to https://sierraweb.com
whatsnew/2.4.rst:691: [redirected permanently] http://speleotrove.com/decimal/ to https://speleotrove.com/decimal/
whatsnew/2.7.rst:1547: [redirected permanently] https://www.openssl.org/docs/man1.0.2/man1/ciphers.html to https://docs.openssl.org/1.0.2/man1/ciphers/
whatsnew/3.11.rst:2045: [redirected permanently] https://pypi.org/project/Pynche/ to https://pypi.org/project/pynche/
whatsnew/3.13.rst:1315: [redirected permanently] https://pypi.org/project/crypt_r/ to https://pypi.org/project/crypt-r/
whatsnew/changelog.rst:19200: [redirected permanently] https://www.openssl.org/docs/man1.1.1/man7/proxy-certificates.html to https://docs.openssl.org/1.1.1/man7/proxy-certificates/
whatsnew/changelog.rst:8710: [redirected permanently] https://www.openssl.org/news/secadv/20230207.txt to https://openssl-library.org/news/secadv/20230207.txt

status "redirected temporarily"

c-api/conversion.rst:13: [redirected temporarily] https://manpages.debian.org/snprintf(3) to https://manpages.debian.org/bookworm/manpages-dev/snprintf.3.en.html
c-api/conversion.rst:19: [redirected temporarily] https://manpages.debian.org/vsnprintf(3) to https://manpages.debian.org/bookworm/manpages-dev/vsnprintf.3.en.html
c-api/conversion.rst:68: [redirected temporarily] https://manpages.debian.org/strtoul(3) to https://manpages.debian.org/bookworm/manpages-dev/strtoul.3.en.html
c-api/conversion.rst:82: [redirected temporarily] https://manpages.debian.org/strtol(3) to https://manpages.debian.org/bookworm/manpages-dev/strtol.3.en.html
library/ctypes.rst:1466: [redirected temporarily] https://manpages.debian.org/dlopen(3) to https://manpages.debian.org/bookworm/manpages-dev/dlopen.3.en.html
library/datetime.rst:2531: [redirected temporarily] https://manpages.debian.org/strftime(3) to https://manpages.debian.org/bookworm/manpages-dev/strftime.3.en.html
library/fcntl.rst:162: [redirected temporarily] https://manpages.debian.org/flock(2) to https://manpages.debian.org/bookworm/manpages-dev/flock.2.en.html
library/fcntl.rst:16: [redirected temporarily] https://manpages.debian.org/fcntl(2) to https://manpages.debian.org/bookworm/manpages-dev/fcntl.2.en.html
library/fcntl.rst:16: [redirected temporarily] https://manpages.debian.org/ioctl(2) to https://manpages.debian.org/bookworm/manpages-dev/ioctl.2.en.html
library/io.rst:1164: [redirected temporarily] https://manpages.debian.org/read(2) to https://manpages.debian.org/bookworm/manpages-dev/read.2.en.html
library/os.rst:1298: [redirected temporarily] https://manpages.debian.org/open(2) to https://manpages.debian.org/bookworm/manpages-dev/open.2.en.html
library/os.rst:1766: [redirected temporarily] https://manpages.debian.org/splice(2) to https://manpages.debian.org/bookworm/manpages-dev/splice.2.en.html
library/os.rst:2009: [redirected temporarily] https://manpages.debian.org/access(2) to https://manpages.debian.org/bookworm/manpages-dev/access.2.en.html
library/os.rst:3801: [redirected temporarily] https://manpages.debian.org/eventfd(2) to https://manpages.debian.org/bookworm/manpages-dev/eventfd.2.en.html
library/os.rst:3958: [redirected temporarily] https://manpages.debian.org/timerfd_create(2) to https://manpages.debian.org/bookworm/manpages-dev/timerfd_create.2.en.html
library/os.rst:4019: [redirected temporarily] https://manpages.debian.org/clock_settime(2) to https://manpages.debian.org/bookworm/manpages-dev/clock_settime.2.en.html
library/os.rst:4019: [redirected temporarily] https://manpages.debian.org/date(1) to https://manpages.debian.org/bookworm/coreutils/date.1.en.html
library/os.rst:4019: [redirected temporarily] https://manpages.debian.org/settimeofday(2) to https://manpages.debian.org/bookworm/manpages-dev/settimeofday.2.en.html
library/os.rst:4019: [redirected temporarily] https://manpages.debian.org/timerfd_settime(2) to https://manpages.debian.org/bookworm/manpages-dev/timerfd_settime.2.en.html
library/os.rst:4049: [redirected temporarily] https://manpages.debian.org/timerfd_gettime(2) to https://manpages.debian.org/bookworm/manpages-dev/timerfd_gettime.2.en.html
library/os.rst:4601: [redirected temporarily] https://manpages.debian.org/pidfd_open(2) to https://manpages.debian.org/bookworm/manpages-dev/pidfd_open.2.en.html
library/os.rst:4608: [redirected temporarily] https://manpages.debian.org/waitid(2) to https://manpages.debian.org/bookworm/manpages-dev/waitid.2.en.html
library/os.rst:5030: [redirected temporarily] https://manpages.debian.org/times(2) to https://manpages.debian.org/bookworm/manpages-dev/times.2.en.html
library/os.rst:5335: [redirected temporarily] https://manpages.debian.org/ptrace(2) to https://manpages.debian.org/bookworm/manpages-dev/ptrace.2.en.html
library/os.rst:620: [redirected temporarily] https://manpages.debian.org/namespaces(7) to https://manpages.debian.org/bookworm/manpages/namespaces.7.en.html
library/os.rst:620: [redirected temporarily] https://manpages.debian.org/setns(2) to https://manpages.debian.org/bookworm/manpages-dev/setns.2.en.html
library/os.rst:831: [redirected temporarily] https://manpages.debian.org/unshare(2) to https://manpages.debian.org/bookworm/manpages-dev/unshare.2.en.html
library/pty.rst:93: [redirected temporarily] https://manpages.debian.org/script(1) to https://manpages.debian.org/bookworm/bsdutils/script.1.en.html
library/resource.rst:307: [redirected temporarily] https://manpages.debian.org/getrusage(2) to https://manpages.debian.org/bookworm/manpages-dev/getrusage.2.en.html
library/resource.rst:43: [redirected temporarily] https://manpages.debian.org/getrlimit(2) to https://manpages.debian.org/bookworm/manpages-dev/getrlimit.2.en.html
library/signal.rst:105: [redirected temporarily] https://manpages.debian.org/pthread_sigmask(3) to https://manpages.debian.org/bookworm/manpages-dev/pthread_sigmask.3.en.html
library/signal.rst:105: [redirected temporarily] https://manpages.debian.org/sigprocmask(2) to https://manpages.debian.org/bookworm/manpages-dev/sigprocmask.2.en.html
library/signal.rst:130: [redirected temporarily] https://manpages.debian.org/abort(3) to https://manpages.debian.org/bookworm/manpages-dev/abort.3.en.html
library/signal.rst:134: [redirected temporarily] https://manpages.debian.org/alarm(2) to https://manpages.debian.org/bookworm/manpages-dev/alarm.2.en.html
library/signal.rst:219: [redirected temporarily] https://manpages.debian.org/signal(7) to https://manpages.debian.org/bookworm/manpages/signal.7.en.html
library/signal.rst:248: [redirected temporarily] https://manpages.debian.org/signal(2) to https://manpages.debian.org/bookworm/manpages-dev/signal.2.en.html
library/signal.rst:412: [redirected temporarily] https://manpages.debian.org/pidfd_send_signal(2) to https://manpages.debian.org/bookworm/manpages-dev/pidfd_send_signal.2.en.html
library/signal.rst:439: [redirected temporarily] https://manpages.debian.org/pthread_kill(3) to https://manpages.debian.org/bookworm/manpages-dev/pthread_kill.3.en.html
library/signal.rst:563: [redirected temporarily] https://manpages.debian.org/siginterrupt(3) to https://manpages.debian.org/bookworm/manpages-dev/siginterrupt.3.en.html
library/signal.rst:605: [redirected temporarily] https://manpages.debian.org/sigpending(2) to https://manpages.debian.org/bookworm/manpages-dev/sigpending.2.en.html
library/signal.rst:620: [redirected temporarily] https://manpages.debian.org/sigwait(3) to https://manpages.debian.org/bookworm/manpages-dev/sigwait.3.en.html
library/signal.rst:646: [redirected temporarily] https://manpages.debian.org/sigwaitinfo(2) to https://manpages.debian.org/bookworm/manpages-dev/sigwaitinfo.2.en.html
library/signal.rst:666: [redirected temporarily] https://manpages.debian.org/sigtimedwait(2) to https://manpages.debian.org/bookworm/manpages-dev/sigtimedwait.2.en.html
library/signal.rst:709: [redirected temporarily] https://manpages.debian.org/head(1) to https://manpages.debian.org/bookworm/coreutils/head.1.en.html
library/socket.rst:1048: [redirected temporarily] https://manpages.debian.org/getnameinfo(3) to https://manpages.debian.org/bookworm/manpages-dev/getnameinfo.3.en.html
library/socket.rst:1132: [redirected temporarily] https://manpages.debian.org/inet(3) to https://manpages.debian.org/bookworm/manpages-dev/inet.3.en.html
library/socket.rst:1528: [redirected temporarily] https://manpages.debian.org/getsockopt(2) to https://manpages.debian.org/bookworm/manpages-dev/getsockopt.2.en.html
library/socket.rst:1616: [redirected temporarily] https://manpages.debian.org/recv(2) to https://manpages.debian.org/bookworm/manpages-dev/recv.2.en.html
library/socket.rst:183: [redirected temporarily] https://manpages.debian.org/vsock(7) to https://manpages.debian.org/bookworm/manpages/vsock.7.en.html
library/socket.rst:1950: [redirected temporarily] https://manpages.debian.org/setsockopt(2) to https://manpages.debian.org/bookworm/manpages-dev/setsockopt.2.en.html
library/socket.rst:555: [redirected temporarily] https://manpages.debian.org/packet(7) to https://manpages.debian.org/bookworm/manpages/packet.7.en.html
library/stat.rst:459: [redirected temporarily] https://manpages.debian.org/chflags(2) to https://manpages.debian.org/bookworm/freebsd-manpages/chflags.2freebsd.en.html
library/termios.rst:14: [redirected temporarily] https://manpages.debian.org/termios(3) to https://manpages.debian.org/bookworm/manpages-dev/termios.3.en.html
library/time.rst:151: [redirected temporarily] https://manpages.debian.org/pthread_getcpuclockid(3) to https://manpages.debian.org/bookworm/manpages-dev/pthread_getcpuclockid.3.en.html
library/time.rst:805: [redirected temporarily] https://manpages.debian.org/tzfile(5) to https://manpages.debian.org/bookworm/manpages/tzfile.5.en.html
library/tkinter.rst:586: [redirected temporarily] https://manpages.debian.org/options(3) to https://manpages.debian.org/bookworm/libcvc4-dev/options.3cvc.en.html
library/tkinter.rst:879: [redirected temporarily] https://manpages.debian.org/bind(3tk) to https://manpages.debian.org/bookworm/tk8.6-doc/bind.3tk.en.html
whatsnew/3.13.rst:1315: [redirected temporarily] https://manpages.debian.org/crypt_r(3) to https://manpages.debian.org/bookworm/libcrypt-dev/crypt_r.3.en.html

status "redirected with Found"

about.rst:6: [redirected with Found] https://www.sphinx-doc.org/ to https://www.sphinx-doc.org/en/master/
distributing/index.rst:14: [redirected with Found] https://packaging.python.org/ to https://packaging.python.org/en/latest/
extending/building.rst:54: [redirected with Found] https://setuptools.readthedocs.io/en/latest/setuptools.html to https://setuptools.pypa.io/en/latest/setuptools.html
extending/extending.rst:25: [redirected with Found] https://cffi.readthedocs.io/ to https://cffi.readthedocs.io/en/stable/
extending/index.rst:28: [redirected with Found] https://cffi.readthedocs.io to https://cffi.readthedocs.io/en/stable/
extending/index.rst:37: [redirected with Found] https://packaging.python.org/guides/packaging-binary-extensions/ to https://packaging.python.org/en/latest/guides/packaging-binary-extensions/
faq/library.rst:181: [redirected with Found] https://www.sphinx-doc.org to https://www.sphinx-doc.org/en/master/
faq/library.rst:553: [redirected with Found] https://groups.google.com/groups?selm=34A04430.CF9@ohioee.com to https://groups.google.com/g/comp.lang.python/c/LJPQHowhlsQ/m/OsoXBiShALIJ
faq/programming.rst:100: [redirected with Found] https://pyinstaller.org/ to https://pyinstaller.org/en/stable/
howto/argparse.rst:820: [redirected with Found] https://babel.pocoo.org/ to https://babel.pocoo.org/en/latest/
howto/functional.rst:1240: [redirected with Found] https://developer.ibm.com/articles/l-prog/ to https://developer.ibm.com/technologies/linux/articles/
howto/functional.rst:1240: [redirected with Found] https://developer.ibm.com/tutorials/l-prog2/ to https://developer.ibm.com/technologies/linux/tutorials/
howto/functional.rst:1240: [redirected with Found] https://developer.ibm.com/tutorials/l-prog3/ to https://developer.ibm.com/technologies/linux/tutorials/
howto/isolating-extensions.rst:205: [redirected with Found] https://github.com/python/cpython/blob/master/Modules/xxlimited.c to https://github.com/python/cpython/blob/main/Modules/xxlimited.c
howto/logging-cookbook.rst:2100: [redirected with Found] https://docs.djangoproject.com/en/stable/topics/logging/#configuring-logging to https://docs.djangoproject.com/en/5.1/topics/logging/
howto/mro.rst:667: [redirected with Found] https://doi.org/10.1145/236337.236343 to https://dl.acm.org/doi/10.1145/236337.236343
howto/pyporting.rst:31: [redirected with Found] https://portingguide.readthedocs.io to https://portingguide.readthedocs.io/en/latest/
howto/unicode.rst:42: [redirected with Found] https://www.unicode.org/versions/latest/#Summary to https://www.unicode.org/versions/Unicode15.1.0/
installing/index.rst:109: [redirected with Found] https://packaging.python.org to https://packaging.python.org/en/latest/
installing/index.rst:118: [redirected with Found] https://packaging.python.org/installing/ to https://packaging.python.org/en/latest/tutorials/installing-packages/
installing/index.rst:136: [redirected with Found] https://packaging.python.org/installing/#requirements-for-installing-packages to https://packaging.python.org/en/latest/tutorials/installing-packages/
installing/index.rst:152: [redirected with Found] https://packaging.python.org/science/ to https://packaging.python.org/en/latest/guides/installing-scientific-packages/
installing/index.rst:241: [redirected with Found] https://packaging.python.org/extensions/ to https://packaging.python.org/en/latest/guides/packaging-binary-extensions/
installing/index.rst:52: [redirected with Found] https://www.pypa.io/ to https://www.pypa.io/en/latest/
installing/index.rst:71: [redirected with Found] https://packaging.python.org/installing/#creating-virtual-environments to https://packaging.python.org/en/latest/tutorials/installing-packages/
library/__main__.rst:167: [redirected with Found] https://pip.pypa.io/ to https://pip.pypa.io/en/stable/
library/ast.rst:2540: [redirected with Found] https://greentreesnakes.readthedocs.io/ to https://greentreesnakes.readthedocs.io/en/latest/
library/ast.rst:2553: [redirected with Found] https://libcst.readthedocs.io/ to https://libcst.readthedocs.io/en/latest/
library/ast.rst:2558: [redirected with Found] https://parso.readthedocs.io to https://parso.readthedocs.io/en/latest/
library/importlib.metadata.rst:16: [redirected with Found] https://setuptools.readthedocs.io/en/latest/pkg_resources.html#entry-points to https://setuptools.pypa.io/en/latest/pkg_resources.html
library/importlib.metadata.rst:16: [redirected with Found] https://setuptools.readthedocs.io/en/latest/pkg_resources.html#metadata-api to https://setuptools.pypa.io/en/latest/pkg_resources.html
library/importlib.metadata.rst:55: [redirected with Found] https://importlib-metadata.readthedocs.io/ to https://importlib-metadata.readthedocs.io/en/latest/
library/importlib.resources.rst:35: [redirected with Found] https://setuptools.readthedocs.io/en/latest/pkg_resources.html to https://setuptools.pypa.io/en/latest/pkg_resources.html
library/importlib.resources.rst:35: [redirected with Found] https://setuptools.readthedocs.io/en/latest/pkg_resources.html#basic-resource-access to https://setuptools.pypa.io/en/latest/pkg_resources.html
library/intro.rst:89: [redirected with Found] https://pyodide.org/ to https://pyodide.org/en/stable/
library/os.path.rst:320: [redirected with Found] https://learn.microsoft.com/windows/dev-drive/ to https://learn.microsoft.com/en-us/windows/dev-drive/
library/platform.rst:295: [redirected with Found] https://www.freedesktop.org/software/systemd/man/os-release.html to https://www.freedesktop.org/software/systemd/man/latest/os-release.html
library/tkinter.rst:45: [redirected with Found] https://www.tkdocs.com/shipman/ to https://tkdocs.com/shipman/
library/tkinter.rst:93: [redirected with Found] https://wiki.tcl-lang.org/37432 to https://wiki.tcl-lang.org/page/Tcl+Package+User+Guide
library/tkinter.ttk.rst:27: [redirected with Found] https://core.tcl.tk/tips/doc/trunk/tip/48.md to https://core.tcl-lang.org/tips/doc/trunk/tip/48.md
library/trace.rst:18: [redirected with Found] https://coverage.readthedocs.io/ to https://coverage.readthedocs.io/en/7.6.1/
library/unittest.mock-examples.rst:1345: [redirected with Found] https://pyhamcrest.readthedocs.io/ to https://pyhamcrest.readthedocs.io/en/latest/
library/unittest.rst:59: [redirected with Found] https://docs.pytest.org/ to https://docs.pytest.org/en/stable/
library/urllib.request.rst:21: [redirected with Found] https://requests.readthedocs.io/en/master/ to https://requests.readthedocs.io/en/latest/
library/venv.rst:56: [redirected with Found] https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/#create-and-use-virtual-environments to https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/
library/wsgiref.rst:32: [redirected with Found] https://wsgi.readthedocs.io/ to https://wsgi.readthedocs.io/en/latest/
library/xmlrpc.client.rst:171: [redirected with Found] http://xmlrpc.scripting.com/spec.html to http://xmlrpc.com/spec.md
license.rst:13: [redirected with Found] https://www.cwi.nl/ to https://www.cwi.nl/en/
license.rst:189: [redirected with Found] http://hdl.handle.net/1895.22/1013 to http://www.handle.net/python_licenses/python1.6.1-2-23-01.html
license.rst:305: [redirected with Found] http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/MT2002/emt19937ar.html to http://www.math.sci.hiroshima-u.ac.jp/m-mat/MT/MT2002/emt19937ar.html
tutorial/whatnow.rst:36: [redirected with Found] https://docs.python.org to https://docs.python.org/3/
using/mac.rst:166: [redirected with Found] https://toga.readthedocs.io to https://toga.readthedocs.io/en/stable/
using/mac.rst:183: [redirected with Found] https://briefcase.readthedocs.io to https://briefcase.readthedocs.io/en/stable/
using/windows.rst:1348: [redirected with Found] https://pythonce.sourceforge.net/ to https://sourceforge.net/projects/pythonce/
whatsnew/2.2.rst:959: [redirected with Found] http://xmlrpc.scripting.com/ to http://xmlrpc.com/
whatsnew/2.5.rst:333: [redirected with Found] https://pylib.readthedocs.io/ to https://pylib.readthedocs.io/en/latest/
whatsnew/2.7.rst:1836: [redirected with Found] https://nose.readthedocs.io/ to https://nose.readthedocs.io/en/latest/
whatsnew/2.7.rst:1836: [redirected with Found] https://pytest.org to https://docs.pytest.org/en/stable/
whatsnew/3.13.rst:1342: [redirected with Found] https://www.pygame.org/ to https://www.pygame.org/news
whatsnew/3.2.rst:2522: [redirected with Found] https://www.mercurial-scm.org/wiki/QuickStart to https://wiki.mercurial-scm.org/QuickStart
whatsnew/3.5.rst:1255: [redirected with Found] https://www.openexr.com to https://openexr.com/en/latest/
whatsnew/3.5.rst:2215: [redirected with Found] https://visualstudio.microsoft.com/en/vs/older-downloads/#visual-studio-2015-and-other-products to https://visualstudio.microsoft.com/vs/older-downloads/
whatsnew/3.7.rst:116: [redirected with Found] https://docs.python.org/fr/ to https://docs.python.org/fr/3/
whatsnew/3.7.rst:116: [redirected with Found] https://docs.python.org/ja/ to https://docs.python.org/ja/3/
whatsnew/3.7.rst:116: [redirected with Found] https://docs.python.org/ko/ to https://docs.python.org/ko/3/
whatsnew/3.9.rst:934: [redirected with Found] https://parso.readthedocs.io/ to https://parso.readthedocs.io/en/latest/
whatsnew/changelog.rst:13310: [redirected with Found] https://www.python.org/dev/peps/pep-0007/#documentation-strings to https://peps.python.org/pep-0007/
whatsnew/changelog.rst:27700: [redirected with Found] https://docs.python.org/zh-cn/ to https://docs.python.org/zh-cn/3/
whatsnew/changelog.rst:9924: [redirected with Found] https://sphinxext-opengraph.readthedocs.io/ to https://sphinxext-opengraph.readthedocs.io/en/latest/

status "timeout"

library/unittest.rst:67: [timeout] http://lists.idyll.org/listinfo/testing-in-python: HTTPConnectionPool(host='lists.idyll.org', port=80): Max retries exceeded with url: /listinfo/testing-in-python (Caused by ConnectTimeoutError(, 'Connection to lists.idyll.org timed out. (connect timeout=30)'))
howto/mro.rst:13: [timeout] https://www.phyast.pitt.edu/~micheles/: HTTPSConnectionPool(host='www.phyast.pitt.edu', port=443): Max retries exceeded with url: /~micheles/ (Caused by ConnectTimeoutError(, 'Connection to www.phyast.pitt.edu timed out. (connect timeout=30)'))

[hugovk]

1) Can we add them to linkcheck_allowed_redirects?

https://github.com/python/cpython/blob/e2f2dc708eae89f41e328501b5ea7c97b8e39907/Doc/conf.py#L534

2) Let's ask @AA-Turner. I wonder if we could somehow use linkcheck_allowed_redirects? It allows regular expressions.