Change FSTRING_END token to include closing quote

we-like-parsers / cpython

Here we work on integrating pegen into CPython; use branch 'pegen'

https://github.com/gvanrossum/pegen

Other

1 stars 0 forks source link

Change FSTRING_END token to include closing quote #210

Closed lysnikolaou closed 1 year ago

lysnikolaou commented 1 year ago

Closes #206.

pablogsal commented 1 year ago

Thanks a lot for working on this! One note, as requested, in the new specification (check https://peps.python.org/pep-0701/#new-tokens) we are now emitting ONLY the quote in the FSTRING_END token and we emit an FSTRING_MIDDLE with the contents before the quote for symmetry.

Of course, as we discussed previously we don't need to follow the specification exactly in the C tokenizer, but if we are changing this may be worth doing. What do you think?

CC: @isidentical Thoughts?

lysnikolaou commented 1 year ago

Oooh, that's right! I completely forgot about that change, I'm sorry. My feeling is we should change the implementation to match the spec. It wouldn't be too hard anyway.