Auto-detection of intersphinx URLs

[mthuurne]

Intersphinx makes documentation much more useful, by being able to deep-link to other documentation. However, it is currently up to the maintainer to add the intersphinx URLs for all dependencies to make this work. Maybe we can simplify this.

pydoctor could auto-detect the intersphinx URL:

• by using known URLs for distributions that we have built-in support for, such as the Python standard library or the various zope packages
• by looking at the Project-URL metadata of installed distributions
• by trying project.readthedocs.io

I dislike it when tools connect to internet services unprompted, so this feature would have to be turned on explicitly through a command line option, such as --intersphinx=auto or --intersphinx-autodetect.

Auto-detection should work together with explicitly passed intersphinx URLs, where explicit URLs always take precedence over auto-detected ones.

[tristanlatr]

by looking at the Project-URL metadata of installed distributions

This sounds hard. Because we need to guess the sub folder when the intersphinx objects inventory is placed, which can be a difficult.

[tristanlatr]

workaround for doc URL lookups from zope.interface:

• The code is here: https://github.com/twisted/twisted/blob/trunk/src/twisted/python/_pydoctor.py
• The tests are here: https://github.com/twisted/twisted/blob/trunk/src/twisted/python/test/test_pydoctor.py

[tristanlatr]

We could use the pypi API to get the project documentation url, from that we should try /objects.inv and /api/objects.inv. The problem lies with packages that don’t have the same python module name as their package name. Or if a package is providing several modules. I’m not quite sure how to handle these relatively common cases.