Use PEP570 syntax in the documentation

[pablogsal]

BPO	37134
Nosy	@gvanrossum, @tim-one, @warsaw, @brettcannon, @rhettinger, @ncoghlan, @vstinner, @serhiy-storchaka, @willingc, @JulienPalard, @bskinn, @pganssle, @pablogsal, @grantjenks, @miss-islington
PRs	python/cpython#13743 python/cpython#13851 python/cpython#13854 python/cpython#13874 python/cpython#13875

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = created_at = labels = ['type-feature', '3.8', 'docs'] title = 'Use PEP570 syntax in the documentation' updated_at = user = 'https://github.com/pablogsal' ``` bugs.python.org fields: ```python activity = actor = 'miss-islington' assignee = 'docs@python' closed = True closed_date = closer = 'pablogsal' components = ['Documentation'] creation = creator = 'pablogsal' dependencies = [] files = [] hgrepos = [] issue_num = 37134 keywords = ['patch'] message_count = 48.0 messages = ['344295', '344357', '344360', '344373', '344379', '344402', '344411', '344457', '344464', '344467', '344473', '344482', '344517', '344518', '344520', '344525', '344527', '344610', '344618', '344621', '344641', '344645', '344646', '344647', '344650', '344652', '344654', '344655', '344656', '344666', '344678', '344739', '344753', '344765', '344767', '344771', '344779', '344781', '344783', '344856', '344857', '344860', '344879', '344884', '344885', '344887', '344889', '344890'] nosy_count = 16.0 nosy_names = ['gvanrossum', 'tim.peters', 'barry', 'brett.cannon', 'rhettinger', 'ncoghlan', 'vstinner', 'docs@python', 'serhiy.storchaka', 'willingc', 'mdk', 'bskinn', 'p-ganssle', 'pablogsal', 'grantjenks', 'miss-islington'] pr_nums = ['13743', '13851', '13854', '13874', '13875'] priority = 'normal' resolution = 'fixed' stage = 'resolved' status = 'closed' superseder = None type = 'enhancement' url = 'https://bugs.python.org/issue37134' versions = ['Python 3.8'] ```

[pablogsal]

In the documentation, there are a lot of mismatches regarding function and method signatures that use positional-only arguments with respect to the docstrings (that properly use PEP-570 syntax for documenting positional-only parameters).

Not that the official syntax for positional-only parameters is accepted (the "/"), the documentation needs to be updated to reflect positional-only parameters in the functions and methods that use them as covered in the PEP document.

[rhettinger]

Do we really want to do this? ISTM, this would be a regression in the readability of the docs. 100% of the users of the docs will be confronted with this syntax which is mostly irrelevant to them.

[rhettinger]

Guido, was it your intention that this show-up everywhere in the docs. I had thought the primary motivation was to make it possible to write pure python code that exactly matched a C API.

ISTM, there is no need to confront every single user of the docs with a notation that is known to be confusing. From a teacher's point of view, pushing this notation everywhere in the docs changes the priority of the topic. Formerly, it was an optional intermediate/advanced topic. But if it is visible everywhere, it becomes an unavoidable day one topic for every single learner (that is if we want them to be able to read and understand the docs).

[gvanrossum]

Please don’t call on just me. If you really don’t want this, call on the Steering Council.

[rhettinger]

Sorry, I thought that as the BDFL delegate for PEP-570, you would be the person to ask. Adding the Steering Council members to the nosy list.

[pablogsal]

Thank you, Raymond, for sharing your concerns regarding this. I am sorry you disagree with this change.

I would want to expose the reasons I think this is important and justified, but please, don't read this as a way of dismissing your concerns; I really respect your opinion and judgment (as an example, I am waiting until you are happy with the API in bpo-17005 even if that meant waiting 2 years until the next release).

1) The docstrings for these functions already show the "/". As Matthias Bussonnier from IPython and Jupyter has expressed multiple times, there are a ton of users that do not read html documentation and rely solely on the output from help (either in Jupyter, IPython or CPython). These users find very strange that the signatures of the docstrings differ when they actually check the html docs. Some examples of the docstrings:

>>> help(zlib.decompress)
    decompress(data, /, wbits=15, bufsize=16384)
    Returns a bytes object containing the uncompressed data.

> help(codecs.register) register(search_function, /) Register a codec search function.

2) This communicates the users more explicitly how to call the function and what to expect when this happens, which is the purpose of documenting signatures in the documentation. We are already documenting keyword-only parameters for the same reasons.

3) This PR is not changing any of the signatures, is just documenting and informing users what the signature is and how it will behave. Is just expressing information. The documentation must be user-friendly, but also needs to be technical.

4) We are already documenting positional-only arguments in multiple other functions in the documentation. This PR just continues doing that. Some examples of the many cases:

https://docs.python.org/3.8/library/functools.html#functools.partialmethod https://docs.python.org/3.8/library/threading.html#threading.excepthook https://docs.python.org/3.8/library/inspect.html#inspect.getcallargs ...

5) I give training sessions to different levels (people that are starting to program, Juniors and Seniors) and they are always confused when the signature of the function in the documentation does not match the docstring or the actual way of calling the function. This also includes when we use constructs like f(x, ...), f(x, [y=None,]), f(K) -> V or similar because they are not valid Python signatures and they do not appear elsewhere. In general, my experience is that people want the signature documented as close as possible to the real signature in the technical documentation.

I wished we could have iterate over this together having a good discussion before involving the steering council, and I am sorry if you were afraid that your concerns would have not been listened to. Sadly, I have not answered before because it was 1:26 in the morning here when you posted your message. I have left the PR open and opened an issue because is the normal workflow and because this allows people to express here their opinion.

I think this change is important, but I also think of working together and respecting and understanding everyone's point of view is much more important. If you think me pushing for this change will impact somehow our ability to work together I will close the PR and the issue.

For now, I won't do anything before the steering council decides what to do.

I apologize to everyone or the wall of text.

[vstinner]

See also bpo-37116.

[rhettinger]

If you think me pushing for this change will impact somehow our ability to work together I will close the PR and the issue.

Not at all. I always enjoy working with you and appreciate the depth of thinking you bring to every task :-)

Here, we just disagree on whether changing the docs will be a net aid or net detriment to end users. My belief is that the docs will be less usable, less intuitive, and less approachable. People using the docs are already alone and in need of help. Cryptic notation doesn't make this task easier. While a person can be trained to read this notation, it is my belief that a person shouldn't have to have training to read the docs.

Unlike other decisions where predicting the future is uncertain, we already have some user testing results because the \ notation was exposed in help() via the argument clinic. The results have not been favorable. AFAICT not one single user has benefitted from seeing something like this output from help(len):

    len(obj, /)
        Return the number of items in a container.

The / is a stumbling block. My unscientific twitter poll had mostly WTF reactions from end-users. This wasn't limited to beginners -- Steve Holden and David Beazley have both found this notation detrimental to communication.

This week I joined a twitter thread where I needed to defend the existing docs. The contention was that docs aren't very usable for end-users and that the existing core devs lacked writing skills and were too interested in technical details rather than plain communication. (I mostly disagree with this, but there is another core dev consistently giving this message in talks all around the world). The essential point here is that there are already usability concerns with the documentation. My belief is that this PR will make the situation worse.

[tim-one]

I haven't looked at anything in this PR, so just popping in to confirm that the first time I saw stuff like:

    len(obj, /)

in the docs I had no idea at all what it was trying to say. I thought it was a typo.

Also the second, third, fourth, ..., times I saw it.

However, because Python didn't _accept_ that syntax in my own function definitions, that may well have made it extraordinarily hard to figure out.

If the syntax is going to be accepted now, that does change things to my eyes. It becomes a question of discoverability then.

I'll note that Googling on

python slash in formal argument list

turns up an excellent Stackoverflow explanation as its top hit for me today, although not if "slash" is replaced with "/".

[brettcannon]

FYI when Guido said "call on the steering council" I think he meant open an issue at https://github.com/python/steering-council/issues as this will get lost otherwise (i.e. Guido already removed himself from the nosy list so this already doesn't have the whole council participating).

Anyway, from my personal view, I think we should use the syntax. It's officially part of the language at this point and so avoiding it in the docs seems odd, like we're saying that we're ashamed of this syntax and you should pretend it doesn't exist when it does and it isn't going anywhere. Otherwise aren't we're forcing users to realize that the function definition in the docs deviates from what is definable in this one instance and that one has to read the full text to know that something is doable syntactically but we're leaving it out of the docs? It adds inconsistency in what the definition line means IMO based on how we have usually chosen to try and express things in that 'def' line in the docs as succinctly as possible to communicate the semantics of calling that function.

And I realize that Raymond's "unscientific twitter poll had mostly WTF reactions from end-users", but that's also sampling from people who have probably never used the syntax in real life or read docs regularly that use it. I've actually been wanting to use the syntax to say "the name isn't important here" as I've had to already close a PR that wanted to tweak documentation for name consistency where it actually didn't matter because the code used positional-only parameters but they weren't explicitly documented as such. Going back and saying "positional-only" to communicate that everywhere that it applies versus using the syntax we have to express that seems unnecessary to me.

Or put another way, how would you want to update:

len(obj)
        Return the number of items in a container.

to properly reflect the fact that *obj is positional-only which is now an official concept in the language? We've been glazing over this for ages in the docs because we didn't have a way of expressing it succinctly and it only touched C-implemented code and we lacked a way to state it in Python, but now we have a way to state this fact. You could add a "The *\obj** parameter is positional-only.", but to me that requires unnecessary reading of the text body if all I'm doing is checking the parameter order (and maybe the first sentence). And I realize we have gotten away with this until now, but as the concept of positional-only parameters grows that will become more of an issue.

Not using / also means that we're deviating from what help() emits, which could cause its own confusion as more people learn about the new syntax.

[willingc]

Tim's message resonated with me. Confusing users is something that I believe that we wish to minimize. I confess that I had a similar reaction as Tim when I saw functions with a trailing /. What I did find helpful was adding to the middle of the parameter list as was done in Serihy's earlier PRs.

Since we are balancing technical accuracy with user confusion (docs and docstrings), I propose the following:

1. Remove the trailing / from 3.8 documentation but leave the / if it occurs in the middle of the parameter list. This increases technical accuracy from pre-3.8 and gives users more time to get comfortable with the change (since end users have found this a confusing area with args kwargs *).

2. Add to the documentation in 3.8 section on positional only parameters a note box that describes that in many/all cases where a / is not specified at the end and no * is found in the parameter list that a trailing slash would be accurate.

3. Give users time to absorb the positional only change in 3.8. Perhaps writing a blog post that explains in detail.

4. Add the trailing / in 3.9 documentation to make consistent with docstrings or improve/create a better directive for function that provides a better accuracy and less confusion. A Sphinx extension may also be made to show a simple user-friendly view and with a click the fully accurate view.

[gpshead]

I agree with Brett and Carol here. This is part of the language syntax. We can educate people to that effect more if desired (I like the blog post idea). People will figure it out.

We've already got high ranked stackoverflow answers related to this when you search: https://stackoverflow.com/questions/24735311/python-what-does-the-slash-mean-in-help-output/ https://stackoverflow.com/questions/28243832/what-is-the-meaning-of-a-forward-slash-in-a-python-method-signature-as-show/

Today, we don't document what is positional only, adding this notation is an improvement because it communicates that detail to anyone interested in understanding it without adding a pile of text to all sorts of function signatures.

[vstinner]

A first enhancement would be to add an index entry for "/" at: https://docs.python.org/dev/genindex-Symbols.html

I'll note that Googling on: "python slash in formal argument list" ...

We have to find a cute name for the "/ operator" in function definition! Like the "walrus operator" for "x := 1" :-D Nobody is able to find "/" in Google. And I'm not sure that "slash" is accurate enough, since it has multiple meaning.

[rhettinger]

I respectfully disagree with logic, "the language now permits this, so we should change all the docs to display it everywhere". That moves it from optional knowledge to mandatory knowledge, from a day ten lesson to a day one lesson, from "once in a while, a user may benefit from being able to mark arguments as positional-only" to the category of "every single user will see this every time they see the docs". There is a huge difference.

My almost daily experience with end-users regarding the \ annotation has been decidedly negative. And my experience as a professional tech writer teaches that "people will just have to figure it out" is not a phrase to live by when it comes to documentation (it is an anti-pattern).

There is no reason that we have to do this to the docs. Please at least go with Carol's suggestion to defer all the trailing \ annotations for at least another release so that you can get more user feedback.

I know some regard The Zen of Python as just poetry, but the "readability counts" and "beautiful is better than ugly" parts mean something. Those key attractors to the language should not be discarded lightly.

And while a blog post might be nice, the odds are that fewer than 1% of Python users will ever see it. Python has millions of users and blog posts are rarely seen by more than tens of thousands. Even then, blog post knowledge is transient and quickly becomes yesterday's news. The entire notion of training away the problem is specious; you really shouldn't have to have training to read documentation.

[pganssle]

I respectfully disagree with logic, "the language now permits this, so we should change all the docs to display it everywhere". That moves it from optional knowledge to mandatory knowledge, from a day ten lesson to a day one lesson, from "once in a while, a user may benefit from being able to mark arguments as positional-only" to the category of "every single user will see this every time they see the docs". There is a huge difference.

A decent part of the reasoning for the PEP was actually to create a standard way to document when arguments to a function are positional-only. By adopting the notation (already used in the numpy documentation IIRC), it makes a concise documentation clear.

Another part of the PEP's reasoning was that this notation is difficult to understand at least partially because it is rarely encountered, so "people don't understand it" as a reason to exclude it from the documentation becomes a bit of a self-fulfilling prophecy.

I am not really an expert in documentation and front-end design, but maybe it would be possible to have some sort of unobtrusive tooltip that activates on signatures that contain positional-only arguments?

[rhettinger]

Another part of the PEP's reasoning was that this notation is difficult to understand at least partially because it is rarely encountered, so "people don't understand it" as a reason to exclude it from the documentation becomes a bit of a self-fulfilling prophecy

We know that isn't true because people already see it frequently in the tooltips for C functions. What I've observed is that after repeated exposures, users learn to treat it as noise and tune it out. I've not found a single instance where a user found it to be helpful.

[c8b49e28-25bb-49e2-b713-2af027a19b3f]

FWIW, I would rather not see the docs littered with "/". I've taught Python to hundreds of professional software engineers over the last five years and in all that time nobody has ever asked when the args need to be positional. It's easy to experiment to find out and it's historically been an implementation detail. I think the number of times people are surprised is far less than the number of times people never notice at all. As Raymond described, this change will elevate the feature to a day-1 topic and it's pretty useless to a day-1 user. This is another step toward making what I see as an unfortunate implementation detail into formal semantics.

[pablogsal]

Thank you, everyone, for expressing clearly your different opinions and concerns. Without disregarding any of the arguments so far I want to make some further clarifications on some points and why I still think this is important.

FWIW, I would rather not see the docs littered with "/".

Although I assume there is not the intention, there is absolutely no reason to be disrespectful to the people that think otherwise. You can express the same idea without implying that the feature is "litter". The same thing applies to "unfortunate implementation" in another of the sentences that was used.

This is another step toward making what I see as an unfortunate implementation detail into formal semantics.

The feature has as much formal semantics as it is possible already: is an official part of the language.

One of the things the PEP makes a lot of emphasis is the current status of the feature and how users see the syntax is precisely caused because of a lack of documentation, exposition and ultimately (and very important) because it was not valid Python syntax. The fact that many people were thinking that is a typo in the documentation or "noise" is precisely because of this and I think having it in the docs is crucial for discovery.

Regarding the usefulness of having the syntax for users: is exactly as useful as knowing that some arguments are keyword-only and those are documented and we did not have any discussion about this. One can disagree and argue that the usefulness of the feature, when some users consider implementing functions that use both syntaxes, is much different between positional-only and keyword-only and that one solve more common problems that the other, but that is irrelevant for people reading the documentation: the relevant thing is that they tell you the restrictions when calling an existing function. And at that point, it does not matter how common something is or how common is fall into the error condition. Also, take into account that there is a serious difference between teaching someone how to react to the syntax (you cannot use keyword parameters for these arguments), which is done almost immediately, and teaching someone when they want to use the syntax themselves on their function. And I want to make clear that I acknowledge that there is a cognitive burden because there are more cases to remember.

I think documenting the trailing "/" is especially important because now users will find very confusing the fact that functions only document the "/" in some places. They may start to believe that a trailing "/" is illegal syntax or that the "/" at the end is optional. This will lead to even more confusion IMHO. This will also perpetuate another thing that the PEP put a lot of focus on solving, which is removing the dissonance between the signature that appears in the documentation, the one in the help() and docstrings and the one that inspect.signature will return. Precisely failing to document all cases will make this even more confusing and will severely alter and bias any feedback that users could provide about any related aspect.

It is possible that some users were thinking that the bare "*" for keyword-only arguments was a typo when it was introduced and maybe they were thinking that the author meant "*args", but we can all agree that that was not a problem. I don't see why this syntax needs to have special treatment regarding that. Another sentence of the ZEN of Python reads "Special cases aren't special enough to break the rules" and the rules at to this point is that there were absolutely no restrictions regarding using new syntax or terminology in the documentation.

I understand the concerns and I take them into account, but for these reasons together with what other core devs are exposing, I think this syntax should be included into the documentation (including trailing "/") as this was one core motivation for the PEP.

[willingc]

I echo Pablo's comment about thoughtful discourse as this is discussed.

For library maintainers that are writing APIs that involve workflows across multiple projects maintained by different people, better information and documentation is very important.

We are balancing the needs of different user groups. An all or nothing approach to documenting / and positional only arguments will not serve the different groups well. What will serve them is coming up with an approach that will work (though not perfectly) for each group. In essence, balancing and compromising are critical.

I suggested an approach earlier which I would appreciate folks give a closer look to its merit.

[pablogsal]

Thank you Carol for your comment! Regarding your proposal, I find it it attractive as a compromise in order to make more gentle the introduction of the feature to users. I really appreciate the effort to balance all the different aspects and concerns.

Ccould you extend your thoughts regarding this concern that I have with respect of not including the trailing / in 3.8 but doing it in 3.9:

I think documenting the trailing "/" is especially important because now users will find very confusing the fact that functions only document the "/" in some places. They may start to believe that a trailing "/" is illegal syntax or that the "/" at the end is optional. This will lead to even more confusion IMHO.

Another minor concern that I have is the fear that we will have a similar discussion in the future when the PR adding the trailing / to 3.9 would be added.

[willingc]

Could you extend your thoughts regarding this concern that I have with respect of not including the trailing / in 3.8 but doing it in 3.9:

Pablo, Sure thing. I believe that a Sphinx extension (possibly existing sphinx-tabs as suggested by @bskinn in the Steering Council issue or one we develop/find) could give us both a "technically accurate" view and "simplified" view. I wasn't sure if this could/would be ready by 3.8. But I think that it is feasible for 3.9. If we can do it for 3.8, fantastic.

If we do the 3.8/3.9 phased approach, I definitely think the 3.8 docs need to be explicit in stating the what/why/when of the / (still searching for a cute name - right now the best I have is "twig" from another name, virgule, for slash and its Latin derivation). Ultimately, 3.9 should either by Sphinx extension or edit contain the "technically accurate" version.

[vstinner]

Python has many parameter types: positional-only, positional-or-keyword, keyword only, etc. Even if we don't add "/", it would be nice to have hints when we put the mouse cursor on a function parameter.

Something like:

function: demo(arg1, *, arg2, arg3=None)

arg1 hint: arg1 is the first mandatory parameter (kind: positional-or-keyword) arg2 hint: arg2 is the second mandatory parameter (kind: keyword-only) arg3 hint: arg3 is an optional parameter (default: None, kind: keyword-only)

Maybe we can also add hints on '*':

• hint: Marker for keyword-only parameter

... I have no idea how complex it would be to implement such Sphinx extenstion, or maybe even implement it in Sphinx.

[rhettinger]

I don't think inclusion in 3.9 should be automatic.

The purpose of the deferment is to evaluate the premise, "the only reason people have found this notation confusing was that it wasn't yet valid syntax available for use in their own code." If that premise turns out to be false, we should think seriously about whether this makes the docs less approachable for everyday users.

This extensive edit of the docs will change the appearance and texture of language from day 1 of a person's exposure to the language. It warrants careful consideration of the costs and benefits. Despite what we might wish to be true, the cost side of the equation is definitely not zero.

[pablogsal]

Pablo, Sure thing. I believe that a Sphinx extension (possibly existing sphinx-tabs as suggested by @bskinn in the Steering Council issue or one we develop/find) could give us both a "technically accurate" view and "simplified" view. I wasn't sure if this could/would be ready by 3.8. But I think that it is feasible for 3.9. If we can do it for 3.8, fantastic.

Thanks for the quick response and for clarifying that. This looks like a very interesting approach. Would the "simplify" view remove keyword-only argument markers as well? What other features should be hidden in this simplified view?

[pablogsal]

If that premise turns out to be false, we should think seriously about whether this makes the docs less approachable for everyday users.

We have not done such a thing for any other feature of the language. I think that since this is now a part of the language, it has all the right to be part of the documentation (possibly in some phased process). The contrary will be extremely unfair to the feature and to the PEP process: not a single other case had to fight separately to be included in the language and the documentation.

And we even have an analogous case: keyword-only argument.

[rhettinger]

Language features don't have rights. People do. :-)

FWIW, there is precedent. We have type annotations in the language but don't use them throughout the docs.

In the end, all that matters is usability. If a notion fails a usability test, then we should adapt accordingly.

When it comes to documentation, we also try to minimize how much a person needs to know in order a mentally parse a piece in isolation. That is a core principle of documentation (the MS Excel docs are an excellent example; a counter-example is Wikipedia's use of the IPA pronunciation notation which is technically superior but is only readable/usable by very few of the readers.).

P.S. I hope you don't come to personally identify with this patch. I'm a big admirer of your work and am already promoting the feature to my 50,000+ twitter follows. In this tracker issue, I hope for us a have a dispassionate, honest evaluation of what makes for the best documentation of the language.

[pablogsal]

FWIW, there is precedent. We have type annotations in the language but don't use them throughout the docs.

Although there is some distance between both cases, I think this is an extremely good point to take into account that I have not thought about.

P.S. I hope you don't come to personally identify with this patch. I'm a big admirer of your work and am already promoting the feature to my 50,000+ twitter follows. In this tracker issue, I hope for us a have a dispassionate, honest evaluation of what makes for the best documentation of the language.

Not at all :). I will be happy whatever solution we end agreeing to, knowing that we are going through a very thorough discussion. But thank you very much for stopping the discussion to check. I actually apologize if any of my responses have been more passionate than it should have been. I (really!) appreciate your words here and the fact that even if you disagree with this change and the cost of the feature regarding the advantages, you are making a lot of effort to explain the feature to users. Thank you very much for caring about the language, users and having a healthy and productive discussion.

[JulienPalard]

FWIW here's my feedback (as a Python teacher and doc guy):

I find that newcomers are good to ignore what they don't understand, so a newcomer exposed to "foo(a, b, /)" will no run away nor cry, he will just ignore the slash and understand that "foo takes two parameters, a and b" and be happy with it.

Then I think that for more advanced people it's nice to have it:

• It's a way to discover it's a valid syntax
• It's a usefull information to use
• It does not take a lot of space
• It's the truth (I mean, displaying "foo(a, b)" for "foo(a, b, /)" is a kind of a lie, I don't like it)

So I'm +1 for using PEP-570 syntax in the documentation.

[vstinner]

Tim Peters: "I haven't looked at anything in this PR, so just popping in to confirm that the first time I saw stuff like: len(obj, /) in the docs I had no idea at all what it was trying to say. I thought it was a typo. (...)"

Carol Willing: "I confess that I had a similar reaction as Tim when I saw functions with a trailing /."

I'm quite sure that many users have same reaction the first time they meet "@decorator", "*args", "def func(*, arg)", etc. PEP-570 introduces a syntax which was still illegal in Python 3.7, so it's normal to be surprised when we meet a new syntax.

In a few years, people will be used to that, but will be surprised by yet another new syntax ;-)

The best we can do is to enhance the documentation to properly document "/", directly in the reference documentation. Sphinx is a great help to add links. And we can maybe extend Sphinx to add even more inline hints.

--

The "/" character was discussed in length in the PEP-570. Many people complained... but nobody succeed to came up with a better syntax. The PEP has been accepted and its now part of the *Python language*.

As explained in length in the PEP-570, the "/" syntax is *not* new. For example, it is used for years in Python docstrings. Paul Ganssle also mentioned that this notation is "already used in the numpy documentation IIRC".

--

I don't think that we must hide some syntax like "*args" or "def func(*, arg)" from the doc, just because people learning Python might be surprised at the first read. I don't think that the reference documentation should be used to learn Python. They are way better resources than that to learn Python. Moreover, it's not really written like a tutorial. We have some https://docs.python.org/dev/howto/ for that.

From my point of view, the *reference documentation is more used by people who are already used to Python and so know the Python syntax. Trying to hide the real API in the *reference documentation sounds weird (wrong) to me. The doc should document the real behavior. The PEP-570 makes is possible, whereas previously, the doc was lying.

If you consider that "len(obj, /)" is a typo, maybe we must fix len to accept obj as a keyword parameter? But they are good reasons why it's a positional only parameter. I don't think that anyone wants to change that. The current trend is more the opposite: convert some positional-or-keyword parameters to positional-only parameters, especially for functions taking a single parameter.

In short, I concur with Brett who wrote:

Anyway, from my personal view, I think we should use the syntax. It's officially part of the language at this point and so avoiding it in the docs seems odd, like we're saying that we're ashamed of this syntax and you should pretend it doesn't exist when it does and it isn't going anywhere. Otherwise aren't we're forcing users to realize that the function definition in the docs deviates from what is definable in this one instance and that one has to read the full text to know that something is doable syntactically but we're leaving it out of the docs? It adds inconsistency in what the definition line means IMO based on how we have usually chosen to try and express things in that 'def' line in the docs as succinctly as possible to communicate the semantics of calling that function.

--

By the way, PR 13743 doesn't modify every single function of the documentation. Only a few files are modified, simply because only few functions accept positional-only parameters. My expectation is more that nobody will notice :-)

[c8b49e28-25bb-49e2-b713-2af027a19b3f]

Pablo, I never intended disrespect toward you personally. And I am sorry my words came across that way. I would rather the feature be one of Python's more esoteric qualities.

I think Carol has described the most reasonable way forward. And in particular, I like this creative idea:

A Sphinx extension may also be made to show a simple user-friendly view and with a click the fully accurate view.

I agree too with Raymond in this point:

I don't think inclusion in 3.9 should be automatic.

As anecdata: I showed the PEP and feature to an intermediate class of 15 professional software engineers today. The first question I got was, "when should I use that in my Python code?" And I think the answer is rarely. We reviewed the motivating cases in the PEP and those made sense to them. But I was left feeling concern that if it were prominent and frequent in the docs then people will be likely to imitate, not least of which through simple copy/paste, what they see there without understanding the C-API and why it is that way.

[serhiy-storchaka]

You need to use the PEP-570 syntax if your function/method have a var-keyword parameter, takes arbitrary keyword arguments, and has other parameters beside the var-keyword parameter ("self" counts). And of course the minimal supported Python version should be 3.8.

In PR 13700 the PEP-570 syntax was added in the documentation of functions that use it in the code (like partial() and getcallargs()) and also in the examples of user code where it is needed (like LRU, Callback and Namespace).

PR 13743 adds it the documentation of functions which do not accept arbitrary keywords, but take some of arguments as positional only and others as positional or keyword. It may be surprising, so I think it is better to document this explicitly.

[3c565929-31be-498b-b929-dd4817d430a0]

First, for anyone interested, there are screenshots and links to docs versions at the SC GH issue (https://github.com/python/steering-council/issues/12#issuecomment-498856524, and following) where we're exploring what the tabbed approach to the PEP-570 docs might look like.

Second, I'd like to suggest an additional aspect of the docs situation for consideration.

Regardless of which specific approach is decided upon here, it seems likely the consensus will be a course of action requiring touching the majority (entirety?) of the stdlib API docs. Currently, all of the API docs are manually curated in the .rst, at minimum for the reasons laid out by Victor in bpo-25461 (https://bugs.python.org/issue25461#msg253381). If there's any chance that the balance of factors has changed such that using autodoc (or similar) *would* now be preferable, IWSTM that now is a good time to have that discussion, so that an autodoc conversion could be done at the same time as the PEP-570 rework.

[brettcannon]

To help short-circuit this discussion and focus on the desired solution, the steering council came to a decision that can be seen at https://github.com/python/steering-council/issues/12#issuecomment-498874939 :

• for Python 3.8 specifically, we think it makes sense to continue to leave the slashes out of the documentation for the all parameters are positional-only case (i.e. "... , /)") and the all parameters are positional-only & keyword-only case (i.e. "..., /, *, ..." and "..., /, *args, ..."). This question can then be revisited for Python 3.9.

• however, we actively want to see them added for the cases where an API has a mixture of positional-only and positional-or-keyword args (i.e. "..., /, ...")

• we'd like to see the rendered documentation for function signatures enhanced to use tooltips to name the positional-only (bare "/"), keyword-only (bare "*"), additional positional args (named "*"), and additional keyword args (named "**") notations (hyperlinking to a glossary entry could also be interesting, but may be too visually noisy based on how the hyperlinks get rendered)

[pablogsal]

Thank you very much, everyone, for sharing your comments, views and concerns and thanks to the steering council for helping to reach a conclusion.

I have updated PR13743 to only modify the functions that fall into the second point of Brett's message as indicated.

[willingc]

New changeset 54edb04aa688c8247570b4f59b5145e3fa417576 by Carol Willing (Pablo Galindo) in branch 'master': bpo-37134: Add PEP-570 notation to the documentation (GH-13743) https://github.com/python/cpython/commit/54edb04aa688c8247570b4f59b5145e3fa417576

[gvanrossum]

IIUC sum() should also have a slash in the middle.

[pablogsal]

New changeset c4c421d619baf2ff2f7e09f55b7ae22b8f863c7b by Pablo Galindo in branch 'master': bpo-37134: Use PEP-570 syntax for sum() (GH-13851) https://github.com/python/cpython/commit/c4c421d619baf2ff2f7e09f55b7ae22b8f863c7b

[miss-islington]

New changeset 23f41a64ea668296fa89e25f3cfa11f63026ecac by Miss Islington (bot) in branch '3.8': bpo-37134: Use PEP-570 syntax for sum() (GH-13851) https://github.com/python/cpython/commit/23f41a64ea668296fa89e25f3cfa11f63026ecac

[3c565929-31be-498b-b929-dd4817d430a0]

Brett, to be clear, this sounds like the tabbed solution is not going to be used at this point? If so, I'll pull down the tabbed docs I'm hosting.

[brettcannon]

@Brian: Probably not. The worry that came up during the steering council meeting was bifurcating the docs could cause confusion as to why it was different, which one was more "right", etc.

[brettcannon]

@Brian: And thanks for the experiment! It was an interesting idea to consider.

[willingc]

@brian, Just to echo Brett's words, the Steering Council appreciated the tabbed prototype and your effort to create it. While we may not use it in this particular case, it's great to keep in mind for other places. Thanks for contributing :D

[3c565929-31be-498b-b929-dd4817d430a0]

:thumbsup:

Glad I happened to be in the right place at the right time to put it together. I'll leave the tabslash repo up for future reference.

[gvanrossum]

Thanks Brian!

[pablogsal]

Thanks Brian for the prototype!

I will close this now. We can reopen if something is missing in the future.

[vstinner]

It seems like the translate() method of bytes and bytearray also needs a "/" in their doc.

I wrote a quick & dirty script to parse C files: ---

import glob

def parse(filename):
    clinic_input = False
    args = False
    positional = False
    func = None
    for line_number, line in enumerate(open(filename)):
        line = line.rstrip()
        if line.startswith("/*[clinic input]"):
            clinic_input = True
            args = False
            positional = False
            func = None
        elif clinic_input and func is None:
            func = line
        elif clinic_input and not args and not line:
            args = True
        elif args and line == "    /":
            positional = True
        elif positional and (line == "    *" or line.startswith("[clinic start generated code]")):
            clinic_input = False
            args = False
            positional = False
            func = None
        elif positional and line:
            print("!!!", filename, line_number, func, repr(line))
        elif not line:
            clinic_input = False
            args = False
            positional = False
            func = None

for filename in glob.glob("*/**.c"):
    parse(filename)

---

Output on the master branch: --- !!! Modules/_struct.c 2224 unpack_from ' buffer: Py_buffer' !!! Modules/_struct.c 2225 unpack_from ' offset: Py_ssize_t = 0' !!! Modules/zlibmodule.c 195 zlib.compress ' level: int(c_default="Z_DEFAULT_COMPRESSION") = Z_DEFAULT_COMPRESSION' !!! Modules/zlibmodule.c 196 zlib.compress ' Compression level, in 0-9 or -1.' !!! Modules/zlibmodule.c 314 zlib.decompress ' wbits: int(c_default="MAX_WBITS") = MAX_WBITS' !!! Modules/zlibmodule.c 315 zlib.decompress ' The window buffer size and container format.' !!! Modules/zlibmodule.c 316 zlib.decompress ' bufsize: ssize_t(c_default="DEF_BUF_SIZE") = DEF_BUF_SIZE' !!! Modules/zlibmodule.c 317 zlib.decompress ' The initial output buffer size.' !!! Modules/zlibmodule.c 744 zlib.Decompress.decompress ' max_length: ssize_t = 0' !!! Modules/zlibmodule.c 745 zlib.Decompress.decompress ' The maximum allowable length of the decompressed data.' !!! Modules/zlibmodule.c 746 zlib.Decompress.decompress ' Unconsumed input data will be stored in' !!! Modules/zlibmodule.c 747 zlib.Decompress.decompress ' the unconsumed_tail attribute.' !!! Objects/bytearrayobject.c 1197 bytearray.translate ' delete as deletechars: object(c_default="NULL") = b\'\'' !!! Objects/bytesobject.c 2080 bytes.translate ' delete as deletechars: object(c_default="NULL") = b\'\'' !!! Python/bltinmodule.c 2282 sum as builtin_sum ' start: object(c_default="NULL") = 0' ---

[pablogsal]

New changeset de76c07a8cd0216c3dce215e4d542e2f45aa022f by Pablo Galindo in branch 'master': bpo-37134: Add PEP-570 notation to the signature of byte{array}.translate (GH-13874) https://github.com/python/cpython/commit/de76c07a8cd0216c3dce215e4d542e2f45aa022f

[miss-islington]

New changeset dba4448c63b687204813a5b04c89dd458d5ac45b by Miss Islington (bot) in branch '3.8': bpo-37134: Add PEP-570 notation to the signature of byte{array}.translate (GH-13874) https://github.com/python/cpython/commit/dba4448c63b687204813a5b04c89dd458d5ac45b

[skirpichev]

It seems, the Python Documentation Editorial Board decided that function signatures in sphinx docs should follow the language syntax in all cases: https://discuss.python.org/t/editorial-board-decisions/58580

@pablogsal, shouldn't this be reopened? There are several closed issues (e.g. https://github.com/python/cpython/issues/101118), but this one seems to be a canonical one.