Fix typo and broken test cases

[yf-yang]

• Change invalid_arguments rules to follow latest Python error message https://github.com/python/cpython/pull/117465
• Fix lint issues
• Fix typos https://github.com/python/cpython/pull/120720

[Original message] Just out of curiosity, will pegen release a minor version based on the change? I'm fixing the spell check error of a project that inherits ParserGenerator.

[yf-yang]

Sure. Thank you for your attention. Do you think I should also submit a PR in cpython repo? It seems too minor.

[lysnikolaou]

It is minor, but you can if you want.

[lysnikolaou]

Could you maybe also look into the test failures here or would you like me to?

[yf-yang]

Just saw it. I'll take care of it.

[yf-yang]

Hmm, I am not so sure

Python 3.10

Python 3.10.9 (main, Jan 11 2023, 09:18:20) [Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ast
>>> ast.parse("f(**a, *b)")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/admin/miniconda3/lib/python3.10/ast.py", line 50, in parse
    return compile(source, filename, mode, flags,
  File "<unknown>", line 1
    f(**a, *b)
      ^^^
SyntaxError: iterable argument unpacking follows keyword argument unpacking

3.11

Python 3.11.9 (main, Apr 19 2024, 11:44:45) [Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ast
>>> ast.parse("f(**a, *b)")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/admin/miniconda3/envs/py311/lib/python3.11/ast.py", line 50, in parse
    return compile(source, filename, mode, flags,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<unknown>", line 1
    f(**a, *b)
           ^
SyntaxError: iterable argument unpacking follows keyword argument unpacking

[yf-yang]

tokenize are yielding same results. Will further investigate later.

[yf-yang]

Maybe related: https://github.com/python/cpython/commit/3d180347ae73119bb51500efeeafdcd62bcc6f78 https://github.com/python/cpython/pull/117444/files

After examine those PR, maybe we just need to update the expected result.

[lysnikolaou]

We probably just need to adjust the column offsets of the SyntaxError depending on version. We do this in other places as well, so should be okay.

[yf-yang]

Before 3.11, both Python and pegen raises 1,3 ~ 1,6 From 3.11, Python raises 1,8~1,9 pegen raises 1,3 ~ 1,6

If I got it right, we should:

• Update pegen to follow latest behavior.
• Correct pos to 1,8 ~ 1,9 and set it to None for Python <= 3.10

Is it right?

[yf-yang]

Will work on the grammar file later.

[yf-yang]

Ref: https://github.com/python/cpython/pull/117465

Follow latest python 3.12 behavior:

Python 3.12.4 | packaged by Anaconda, Inc. | (main, Jun 18 2024, 10:14:12) [Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> f(**a, *b)
  File "<stdin>", line 1
    f(**a, *b)
         ^^^^
SyntaxError: iterable argument unpacking follows keyword argument unpacking

Ready to go.

[yf-yang]

🤨 What's that failure? My local py3.8.19 passes all the tests

[yf-yang]

@lysnikolaou Why was this one closed? Are we planning to directly sync cpython code sometime later?

[lysnikolaou]

It was closed because of Github autoclosing referenced issues/PRs when there's the word fix in the PR description. Reopening.

[yf-yang]

OK, would you please take a look at the broken 3.8 test suites? I don't quite understand it.

[MatthieuDartiailh]

Black complains about the formatting of some files. Fixing it may be as simple as running it.

[yf-yang]

Oh yes, thank you. I wasn't familiar with the ci logs 😅.