search

we-like-parsers / pegen

PEG parser generator for Python
https://we-like-parsers.github.io/pegen/
MIT License
155 stars 33 forks source link

python.gram: SyntaxError column numbers to match cpython 3.10 #48

Closed edemaine closed 2 years ago

edemaine commented 2 years ago

As suggested by @MatthieuDartiailh in this comment, this is a completion of #41 (which includes @MatthieuDartiailh original commits) that includes all the row/column tests, as well as a fix to my issue #47.

MatthieuDartiailh commented 2 years ago

Looks great ! Thanks for taking the time to do that.

Since the idea is to match CPython it may make sense to have CPython parse the same test code and validate our error syntax against the CPython one. WDYT ?

cfbolz commented 2 years ago

hi everyone! I am considering to improve a few things in python.gram but don't want to tread on anybody's else's effort. Should I help out with the failing tests here?

(I was wondering whether the reason for the test failures is that python_generator.py does not seem to special-case the invalid_-prefixed rules in the same way that CPython's c_generator.py does it?)

edemaine commented 2 years ago

I'd definitely appreciate the help with fixing these tests! Or rather, fixing python.gram so that these tests which should pass actually do pass. (Though not sure whether @MatthieuDartiailh is currently working on it.)

And we should make the comparison to Python only run on 3.10 (and maybe even a specific minor version?). Incidentally, CI should probably be updated to use 3.10.0.

MatthieuDartiailh commented 2 years ago

I am not working on this at the moment no.

pablogsal commented 2 years ago

should probably be updated to use 3.10.0.

Done in https://github.com/we-like-parsers/pegen/pull/50

MatthieuDartiailh commented 2 years ago

If nobody is working on this I will try to pick this up again soon now that my proof of concept for using pegen in a project I maintain is ready. (https://github.com/nucleic/enaml/pull/474).

I will probably first look into using the two passes error reporting mechanism used in the C parser.

cfbolz commented 2 years ago

I didn't get around to it sorry :-(

MatthieuDartiailh commented 2 years ago

With #41 merged I believe everything this addressed has been fixed so closing. Feel free to reopen if the need arises.