Support async/await expression from Python 3.5

[dahlia]

See also https://docs.python.org/3.5/whatsnew/3.5.html#pep-492-coroutines-with-async-and-await-syntax.

[bjpop]

New grammar

Main new rules:

Difference in grammar between 3.4 and 3.5:

24c24,26
< decorated: decorators (classdef | funcdef)
---
> decorated: decorators (classdef | funcdef | async_funcdef)
> 
> async_funcdef: ASYNC funcdef
25a28
> 
43c46
< augassign: ('+=' | '-=' | '*=' | '/=' | '%=' | '&=' | '|=' | '^=' |
---
> augassign: ('+=' | '-=' | '*=' | '@=' | '/=' | '%=' | '&=' | '|=' | '^=' |
68c71,72
< compound_stmt: if_stmt | while_stmt | for_stmt | try_stmt | with_stmt | funcdef | classdef | decorated
---
> compound_stmt: if_stmt | while_stmt | for_stmt | try_stmt | with_stmt | funcdef | classdef | decorated | async_stmt
> async_stmt: ASYNC (funcdef | with_stmt | for_stmt)
92c96
< # sake of a __future__ import described in PEP 401
---
> # sake of a __future__ import described in PEP 401 (which really works :-)
100c104
< term: factor (('*'|'/'|'%'|'//') factor)*
---
> term: factor (('*'|'@'|'/'|'%'|'//') factor)*
102c106,107
< power: atom trailer* ['**' factor]
---
> power: atom_expr ['**' factor]
> atom_expr: [AWAIT] atom trailer*
114,115c119,122
< dictorsetmaker: ( (test ':' test (comp_for | (',' test ':' test)* [','])) |
<                   (test (comp_for | (',' test)* [','])) )
---
> dictorsetmaker: ( ((test ':' test | '**' expr)
>                    (comp_for | (',' (test ':' test | '**' expr))* [','])) |
>                   ((test | star_expr)
>                    (comp_for | (',' (test | star_expr))* [','])) )
119,121c126,127
< arglist: (argument ',')* (argument [',']
<                          |'*' test (',' argument)* [',' '**' test] 
<                          |'**' test)
---
> arglist: argument (',' argument)*  [',']
> 
124c130,141
< argument: test [comp_for] | test '=' test  # Really [keyword '='] test
---
> # "test '=' test" is really "keyword '=' test", but we have no such token.
> # These need to be in a single rule to avoid grammar that is ambiguous
> # to our LL(1) parser. Even though 'test' includes '*expr' in star_expr,
> # we explicitly match '*' here, too, to give it proper precedence.
> # Illegal combinations and orderings are blocked in ast.c:
> # multiple (test comp_for) arguements are blocked; keyword unpackings
> # that precede iterable unpackings are blocked; etc.
> argument: ( test [comp_for] |
>             test '=' test |
>             '**' test |
>             '*' test )
> 

[garetht]

Implemented here https://github.com/bjpop/language-python/pull/38

[LucianU]

Judging by the last comment, this can be closed, no?

[garetht]

Yup.