Fix converting AST expression to string and optimize parenthesis

[serhiy-storchaka]

BPO	33475
Nosy	@ambv, @serhiy-storchaka
PRs	python/cpython#6774 python/cpython#6927

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = created_at = labels = ['interpreter-core', '3.8', 'type-bug', '3.7'] title = 'Fix converting AST expression to string and optimize parenthesis' updated_at = user = 'https://github.com/serhiy-storchaka' ``` bugs.python.org fields: ```python activity = actor = 'serhiy.storchaka' assignee = 'none' closed = True closed_date = closer = 'serhiy.storchaka' components = ['Interpreter Core'] creation = creator = 'serhiy.storchaka' dependencies = [] files = [] hgrepos = [] issue_num = 33475 keywords = ['patch'] message_count = 4.0 messages = ['316439', '316881', '316887', '317177'] nosy_count = 2.0 nosy_names = ['lukasz.langa', 'serhiy.storchaka'] pr_nums = ['6774', '6927'] priority = 'normal' resolution = 'fixed' stage = 'resolved' status = 'closed' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue33475' versions = ['Python 3.7', 'Python 3.8'] ```

[serhiy-storchaka]

The proposed patch fixes bugs in ast_unparse.c, makes it generating cleaner string representation, and makes the code cleaner.

Known fixed bugs:

1. Extended slices crash Python. "s[a, b:c]".

2. 1-tuples produce illegal syntax: "(a,)" => "(, a)".

3. Lambdas in f-strings produce illegal syntax. "f'{(lambda x: x)}'" => "f'{lambda x: x}'".

4. Some expressions that need parenthesis don't have them. E.g. "lambda x: (x, x)" => "lambda x: x, x".

5. Generators and yield expression always must be surrounded with parenthesis. "(x for x in y)" => "x for x in y", "(yield)" => "yield".

6. Top-level tuples must be surrounded with parenthesis. "(a, b)" => "a, b".

Produced string representation is now more clean and almost not contains redundant parenthesis.

"(a + b) (c + d)" => "(a + b) (c + d)" "(a b) + (c d)" => "a b + c d" "(a b) (c d)" => "a b (c d)" "(a b) (c d)" => "(a b) c d" "[(a + b)]" => "[a + b]" "[(i 2) for i in range(1, (a + 1))]" => "[i 2 for i in range(1, a + 1)]"

Maybe other bugs were fixed in process.

And finally I use macros for simple repeated fragments of code like

        if (-1 == append_charp(writer, str)) {
            return -1;
        }

This made the meaningful code much shorter and saved around 250 lines of code.

[ambv]

New changeset 64fddc423fcbe90b8088446c63385ec0aaf3077c by Łukasz Langa (Serhiy Storchaka) in branch 'master': bpo-33475: Fix and improve converting annotations to strings. (GH-6774) https://github.com/python/cpython/commit/64fddc423fcbe90b8088446c63385ec0aaf3077c

[serhiy-storchaka]

There are still problems with f-strings like f'''{"'"}''' => f'{"\'"}' (SyntaxError: f-string expression part cannot include a backslash). But this is a separate hard issue (bpo-33552).

[serhiy-storchaka]

New changeset b32f8897eafcf335f9e9899c5e622a0b27679437 by Serhiy Storchaka in branch '3.7': [3.7] bpo-33475: Fix and improve converting annotations to strings. (GH-6774). (GH-6927) https://github.com/python/cpython/commit/b32f8897eafcf335f9e9899c5e622a0b27679437