Improve Python Language Reference based on [Köhl 2020]

[gvanrossum]

BPO	46754
Nosy	@gvanrossum, @JelleZijlstra, @corona10, @brandtbucher

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = None created_at = labels = [] title = 'Improve Python Language Reference based on [K\xc3\xb6hl 2020]' updated_at = user = 'https://github.com/gvanrossum' ``` bugs.python.org fields: ```python activity = actor = 'JelleZijlstra' assignee = 'none' closed = False closed_date = None closer = None components = [] creation = creator = 'gvanrossum' dependencies = [] files = [] hgrepos = [] issue_num = 46754 keywords = [] message_count = 4.0 messages = ['413275', '413863', '413865', '413873'] nosy_count = 4.0 nosy_names = ['gvanrossum', 'JelleZijlstra', 'corona10', 'brandtbucher'] pr_nums = [] priority = 'normal' resolution = None stage = None status = 'open' superseder = None type = None url = 'https://bugs.python.org/issue46754' versions = [] ```

[gvanrossum]

In https://arxiv.org/pdf/2109.03139.pdf ("M Köhl, An Executable Structural Operational Formal Semantics for Python, Master Thesis 2020 Saarland University) there are some observations on cases where the Language Reference (referred to as PLR) is ambiguous or incorrect.

Somebody should go over the thesis, collect the issues, and then we can update the language reference.

See also https://github.com/faster-cpython/ideas/issues/208#issuecomment-1039612432

[gvanrossum]

A few examples of issues brought up by Kohl:

• While the PLR explicitly states that “x \< y calls x.__lt(y)” [20, §3.3.1.] this is actually false. There are cases where x \< y does not call x.__lt(y) and there are other cases where x.__lt__(y) is called but more than that happens.

• If no expression is provided, the PLR states that “the last exception that was active in the current scope” should be re-raised. Unfortunately, the PLR stays unspecific on what it means for an exception to be “the last exception that was active in the current scope. [...] Instead, raise re-raises the exception that *is active* in the respective execution context

(Perhaps unrelated, but indicative of how out of date the PLR is: in executionmodel.rst there's still a mention of and even an index entry for restricted execution, a feature that was removed in some early Python 2 release.)

[gvanrossum]

Another:

• The description of this name binding and resolution process in the PLR [20, §4.2.] is unfortunately not particularly clear.

(I found this to be the case too, and wrote up what I found: https://gvanrossum.github.io/formal/scopesblog.html Hopefully it matches what Kohl derived.)

[JelleZijlstra]

Some specific points from the document:

"While the PLR explicitly states that “x \< y calls x.__lt__(y)” [20, 3.3.1.] this is actually false." They had to read the implementation to actually figure out how this works.

"If no expression is provided, the PLR states that “the last exception that was active in the current scope” should be re-raised. Unfortunately, the PLR stays unspecific on what it means for an exception to be “the last exception that was active in the current scope.”"

"The description of this name binding and resolution process in the PLR [20, 4.2.] is unfortunately not particularly clear"

"While generator objects seem not to be specified as part of the PLR, the documentation of the inspect module5 describes them."

(I didn't read the whole thing, but searched for "PLR".)

Here's a few similar things I noticed while reading through the reference recently:

• https://docs.python.org/3.10/reference/datamodel.html#the-standard-type-hierarchy is supposed to be a list of primitive types but has a few problems:
  • Lists "numbers.Number" as a standard type, when it's an ABC that's really not part of the core language
  • Lists "I/O objects" as a kind of standard object, but there are many kinds of file-like objects, and I'd argue they're not part of the core language, but of the standard library.
  • On the other hand, some builtin types (range, map, filter, zip) are not listed.
• https://docs.python.org/3.10/reference/expressions.html#subscriptions doesn't actually give the rules about when __class_getitem is called vs. __getitem. That's in https://docs.python.org/3.10/reference/datamodel.html#class-getitem-versus-getitem.
• The text at https://docs.python.org/3.10/reference/expressions.html#calls doesn't cover positional-only parameters.

[gvanrossum]

Is anyone interested in picking this up? The best approach is probably making one or a few small improvements at a time and getting several (many?) small docs PRs merged.