Give proper link in help(idlelib/turtledemo/tkinter.sub/test_*/?)

[ff014e15-a125-4b8d-b08d-d53768e316b2]

BPO	40051
Nosy	@smontanaro, @terryjreedy, @merwok, @bitdancer, @vadmium, @zware, @wyz23x2, @Joshuah143
PRs	python/cpython#29419 python/cpython#29459

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = None created_at = labels = ['type-bug', 'library', '3.11'] title = 'Give proper link in help(idlelib/turtledemo/tkinter.sub/test_*/?)' updated_at = user = 'https://github.com/wyz23x2' ``` bugs.python.org fields: ```python activity = actor = 'eric.araujo' assignee = 'docs@python' closed = False closed_date = None closer = None components = ['Library (Lib)'] creation = creator = 'wyz23x2' dependencies = [] files = [] hgrepos = [] issue_num = 40051 keywords = ['patch'] message_count = 15.0 messages = ['364917', '365232', '365246', '368021', '368044', '368102', '374998', '404542', '404545', '404825', '405765', '405767', '405768', '405922', '405923'] nosy_count = 9.0 nosy_names = ['skip.montanaro', 'terry.reedy', 'eric.araujo', 'r.david.murray', 'docs@python', 'martin.panter', 'zach.ware', 'wyz23x2', 'Joshuah143'] pr_nums = ['29419', '29459'] priority = 'normal' resolution = None stage = 'test needed' status = 'open' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue40051' versions = ['Python 3.11'] ```

[ff014e15-a125-4b8d-b08d-d53768e316b2]

When typing this in shell:
>>> import lib2to3
>>> help(lib2to3)

The output contains this link: --snip-- MODULE REFERENCE https://docs.python.org/3.8/library/lib2to3 \<--

The following documentation is automatically generated from the Python

--snip--

But when you access it, 404! This works: https://docs.python.org/3.8/library/2to3.html#module-lib2to3

Please change it. Thanks!

[terryjreedy]

Lib/lib2to3 is a directory with __init__.py containing ### empty Unlike most python-coded modules, there is no Doc/library/lib2to3.rst and hence no generated lib2to3.html. Instead, there is a 2to3.rst and 2to3.html. In the module index, module xyz is usually linked to .../library/xyz.html#module-xyz. But lib2to3 is somewhere linked instead to .../library/2to3.html#module-lib2to3.

As implied by the changed title, this issue is not at all unique to lib2to3. Just as lib2to3 implements the command line app 2to3, with 2to3.rst, idlelib implements IDLE with idle.rst. Turtle demo is domumented within turtle.rst. Most tkinter submodule have no doc other than a mention within tkinter.rst. These could perhaps have a # module-tkinter.xyz target added. (I am not familiar with exactly how.) But there are also the numerous test modules that have no doc, and the generated doc for these is less useful than the code itself.

Possible solutions framed in terms of lib2to3:

1. Manual redirection: add lib2to3.rst and hence lib2to3.html with the correct url, possibly with additional text. But this is not the only module documented within a file with another name, and would not work for things like test files that should not be documented.

2. Auto redirection: persuade whoever manages docs.python.org to add a redirection for lib2to3. Fragile.

3. Hardcode the exception in the help output generation. The latter is generated by pydoc.help. The module doc location under MODULE REFERENCE is generated by pydoc.Doc.getdocloc. This function already has a couple of (probably obsolete) tuples of exceptions, so adding more seems OK.

The oddity is that .html is added if and only if the doc location does not start with 'http'. lines 414-417.I think that this is backwards. It only works because docs.python.org adds missing .html, even to "https://docs.python.org/3.8/library/2to3#module-lib2to3", which is why the bug? has not been noticed. Note that by default, docloc starts with 'https://', so by default, .html is omitted.

Skip, David, and Martin, you have all modified this part of the code. Do any of you disagree that there is a bug?

[ff014e15-a125-4b8d-b08d-d53768e316b2]

My opinion: I think No.2 makes more sense to users that visit the docs directly by https://docs.python.org/3.8/library/lib2to3.html; they will copy the "docs.python.org/version/library/modulename.html" format from other modules. But I also agree it's fragile. No.3 is good too, according to me. Can use it if No.2 is too fragile.

[ff014e15-a125-4b8d-b08d-d53768e316b2]

Patch?

[terryjreedy]

Not until decision made. And not be me until I have my development machine running.

[ff014e15-a125-4b8d-b08d-d53768e316b2]

OK.

[ff014e15-a125-4b8d-b08d-d53768e316b2]

Ping? Which of the 3 should we choose?

[zware]

This one got caught up in the purge of 2to3 issues, but it's about more than just 2to3. Reopening.

[terryjreedy]

As the title suggested, this is a generic help(module) issue that involves several library modules, including idlelib. It just happened to be reported for 2to3. For example, the generated text for import idlelib; help(idlelib) includes MODULE REFERENCE https://docs.python.org/3.11/library/idlelib.html It should be .../idle.html.

The holdup is that I was not then willing to unilaterally decide which of the 3 possible solutions I listed should be pursued. Maybe just 3, maybe a mix of 1 and 3.

[merwok]

Option 3 seems simple and ok to me.

[zware]

Option 4: generate a list of modules (or a mapping of module names to documentation files) that have documentation when generating Lib/pydoc_data/topics.py, and teach pydoc.help to not include the link when it knows the module doesn't have documentation.

This is a much more complete solution that also avoids creating links for private modules (see bpo-45717), but is significantly more effort to implement.

[a9d9ee8b-5ac3-4977-91f4-c5e83c4630d6]

I think that option 4 would work best but instead of putting any logic in (Lib/pydoc_data/topics.py)[https://github.com/python/cpython/blob/main/Lib/pydoc_data/topics.py] we could put the logic for working with private modules as well and other edge cases in (Lib/pydoc.py)[https://github.com/python/cpython/blob/main/Lib/pydoc.py]. It might make future changes easier.

[zware]

This actually turned out to be less effort than I expected (though I still should have been asleep 2 hours ago), so I've gone ahead and opened python/cpython#29419 to implement what I suggested.

[terryjreedy]

PR-29419 solves the issue in bpo-45717 of not giving a link for * modules with no doc. It does not solve this issue of giving the proper link for module that need non-standard links, which typically need a '#' suffix, as in '.../library/2to3.html#module-lib2to3'. I suggest to continue calculating standard links as done now as the precalculation needs much space for a trivial time saving. Restrict the new dict to non-standard links. They can be calculated, with the needed suffixes, as they are now for the index. The link logic would be to try the lookup first and if module name does not start with '\', calculate the link.

I will add an idlelib section to idle.rst so that 'idlelib' appears in the module index and get entered into the exceptions dict.

I don't see how PR-29459 get linked here as it has no reference to this issue. So I will ignore it at least for know.

[merwok]

I think that second PR was linked using the GitHub PR link field present in the bug comment form.