document PEP 448: unpacking generalization

[benjaminp]

BPO	24136
Nosy	@warsaw, @terryjreedy, @pfmoore, @vstinner, @tjguk, @benjaminp, @ezio-melotti, @bitdancer, @berkerpeksag, @vadmium, @zware, @zooba, @NeilGirdhar, @JelleZijlstra, @supriyantomaftuh
Files	whatsnew.diff wn2.diff wn2.diff reference_calls_syntax_update.diff: Update of the function call expression syntax. replace_sequence_with_iterable.diff: Replace "sequence" with "iterable" in regard to the assigned object in the assignment statement. issue24136-expressions.patch: revised patch for call syntax in the language reference unpacking-doc.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = created_at = labels = ['easy', 'type-feature', 'docs'] title = 'document PEP 448: unpacking generalization' updated_at = user = 'https://github.com/benjaminp' ``` bugs.python.org fields: ```python activity = actor = 'martin.panter' assignee = 'docs@python' closed = True closed_date = closer = 'martin.panter' components = ['Documentation'] creation = creator = 'benjamin.peterson' dependencies = [] files = ['39370', '39561', '39562', '39918', '39919', '43302', '43339'] hgrepos = [] issue_num = 24136 keywords = ['patch', 'easy'] message_count = 19.0 messages = ['242668', '243160', '244417', '246381', '246385', '246393', '246446', '246452', '246485', '246675', '261616', '263262', '267790', '267792', '267822', '268183', '268290', '268312', '268337'] nosy_count = 19.0 nosy_names = ['barry', 'terry.reedy', 'paul.moore', 'vstinner', 'tim.golden', 'benjamin.peterson', 'ezio.melotti', 'r.david.murray', 'docs@python', 'python-dev', 'berker.peksag', 'martin.panter', 'zach.ware', 'steve.dower', 'NeilGirdhar', 'moigagoo', 'JelleZijlstra', 'supriyanto maftuh', 'supriyantomaftuh'] pr_nums = [] priority = 'high' resolution = 'fixed' stage = 'resolved' status = 'closed' superseder = None type = 'enhancement' url = 'https://bugs.python.org/issue24136' versions = ['Python 3.5', 'Python 3.6'] ```

[benjaminp]

PEP-448 has been implemented (bpo-2292), but the documentation hasn't been updated. Updating the documentation will improve looking through

Doc/reference/*

and making sure the documentation (and grammar) for calls and assignments is updated for PEP-448's new syntax.

I'm marking this as "easy" because it might be a good first bug.

[2c693aaf-b711-4c74-9243-f6794805e1ba]

Just updated the "what's new". Also, thank you for adding my name to Misc/Acks. Should we also add Joshua Landau's name? He helped me quite a bit with the implementation, and he wrote the PEP.

[2c693aaf-b711-4c74-9243-f6794805e1ba]

Simplified functools.partial documentation.

[40a76aee-3420-4ea6-ae96-c95c67475179]

Hi!

I'd like to update the docs with the examples of the new syntax usage. This is my first contribution to the Python docs, so I'd like to ask for some assistance.

I'm going to start with adding an example to the tutorial (https://docs.python.org/3.5/tutorial/introduction.html#lists). I wanted to demonstrate the new syntax with string too (https://docs.python.org/3.5/tutorial/introduction.html#strings), but it turned out to produce somewhat unexpected results:

>>> s = 'And now'
>>> first, *rest = s
>>> # I expected it to be synonymous
>>> # to ``first, rest = s[0], s[1:]``
>>> # ``first`` is expected to be 'A',
>>> # ``rest`` is expected to be 'nd now'.
>>> # ``first`` is 'A', as expected:
>>> first
'A'
>>> # But ``rest`` is implicitly turned into a list:
>>> rest
['n', 'd', ' ', 'n', 'o', 'w', ' ', 'f', 'o', 'r', ' ', 's', 'o', 'm', 'e', 't', 'h', 'i', 'n', 'g', ' ', 'c', 'o', 'm', 'p', 'l', 'e', 't', 'e', 'l', 'y', ' ', 'd', 'i', 'f', 'f', 'e', 'r', 'e', 'n', 't']

Is this behavior intended? Why wasn't first converted into ['A'] as well? Am I just not supposed to use the new unpacking with strings?

Thanks, Konstantin

[vadmium]

Yes I think it is expected and documented that the leftovers are turned into a list. See \https://docs.python.org/3.5/reference/simple_stmts.html#index-6\. I originally had similar confusion, expectating the starred target to become a tuple, because people often use tuple-like syntax, but:

>>> generator_expression = (2**i for i in range(4))
>>> (one, *a_list, eight) = generator_expression
>>> a_list  # Not a tuple!
[2, 4]

One thing in the section I linked above that should also be fixed is that the assigned object may be any iterable, not just a sequence.

About changing the tutorial, just be careful you don’t add unnecessary complication too early. The original * and ** syntax for function parameters is not mentioned until \https://docs.python.org/3.5/tutorial/controlflow.html#more-on-defining-functions\. Later, argument unpacking: \https://docs.python.org/3.5/tutorial/controlflow.html#unpacking-argument-lists\. Assignment unpacking doesn’t seem to mentioned at all (not that I am saying it should be). It might be higher priority to update the main reference documentation first.

[40a76aee-3420-4ea6-ae96-c95c67475179]

@vadmium thanks for the assistance! I'll kick off with the reference then.

P.S. Am I the only one who doesn't receive any emails from the tracker? I never got the registration link or a follow-up notification from this issue. Am I missing something?

P.P.S. I'm not yet familiar with the local etiquette, so please forgive me if I'm unintentionally breaking some rules. Is @mentioning OK?

[2c693aaf-b711-4c74-9243-f6794805e1ba]

I don't receive emails from the tracker anymore either and I have no idea why that is.

[vadmium]

FWIW, I still emails from the tracker, even the ones with my own comments and changes. All I can suggest is check the address you have set, check for spam, etc.

I don’t @mentioning will do anything here. But as long as the person is in the nosy list they _should_ get an email (in theory :).

[2c693aaf-b711-4c74-9243-f6794805e1ba]

Copied from closed bpo-24240:

Since Grammar/Grammar relies on semantic postprocessing in ast.c, it would be nice to have an update of the (human readable) Grammar in the language reference docs.

[40a76aee-3420-4ea6-ae96-c95c67475179]

I've updated the Calls syntax reference in reference/expressions and the assignment object description in reference/simple_stmts.

Please tell me if I'm generally doing OK. If I'm not, please guide me to the right direction.

[terryjreedy]

It is now 10 months and 2 releases since the rather large code patch. Documenting the extensive changes does not seem easy to me ;-). Certainly, a beginner needs feedback.

[berkerpeksag]

supriyanto maftuh,st, please don't play with tracker items.

[JelleZijlstra]

Here's what I found reviewing what needs to be done here:

• Neil's What's New patch apparently made it into the 3.5 release notes.
• moigagoo's two patches haven't been committed. The text of both looks ok to me, but the patches don't apply cleanly.

Other areas that still need updates:

• https://docs.python.org/3/reference/expressions.html#list-displays and the sections for set, dict, and tuple displays need to mention PEP-448-style syntax.

[JelleZijlstra]

This updates reference_calls_syntax_update.diff. The previous patch's grammard had a mistake; it was missing commas between arguments.

I believe all other patches in this diff are now obsolete.

[vadmium]

Thanks for helping with this Jelle.

The documentation of unpacking sequences vs iterables was adjusted in 3.6 as part of bpo-23275. I guess part of revision 8a0754fed986 should be extracted to 3.5 as well.

Looking at the function call syntax, positional and starred should be optional. I don’t think your syntax would allow print(file=stderr).

[vadmium]

Here is a new patch that also updates the documentation for list etc displays as well as function calls. Let me know what you think.

The 3.5 What’s New notes were written separately; Neil’s patch was never applied. But I have rescued his update for functools.partial() in my new patch.

While experimenting with the current behaviour, I found some surprising inconsistencies. The following syntaxes are allowed:

>> x, *y >> a = x, *y >> f"{x, *y}" # New in 3.6 >> async def f(): await x, *y

But the following all produce “SyntaxError: invalid syntax”:

>> a += x, *y >> eval("x, *y") >> def f(): return x, *y >> def f(): yield x, *y >> for i in x, *y: ...

Also, the expressions allowed for unpacking in general are more limited than in function calls:

>>> f(x, *y == z)  # Allowed
>>> (x, *y == z)
SyntaxError: invalid syntax

[JelleZijlstra]

Thanks for writing a better patch. The patch looks good to me and it builds correctly.

[1762cc99-3127-4a62-9baf-30c3d0f51ef7]

New changeset a7e04b4e51b2 by Martin Panter in branch '3.5': Issue bpo-24136: Document generalized unpacking, PEP-448 https://hg.python.org/cpython/rev/a7e04b4e51b2

New changeset 4cf3389cd8e6 by Martin Panter in branch 'default': Issue bpo-24136: Merge unpacking doc from 3.5 https://hg.python.org/cpython/rev/4cf3389cd8e6

New changeset 2c10f0e92256 by Martin Panter in branch 'default': Issue bpo-24136: Adjust f-strings doc for interable unpacking https://hg.python.org/cpython/rev/2c10f0e92256

[vadmium]

Thanks for the review. I committed my patch in the hope that it makes it into 3.5.2, but if people want to suggest further improvements etc that is okay.