search

we-like-parsers / cpython

Here we work on integrating pegen into CPython; use branch 'pegen'
https://github.com/gvanrossum/pegen
Other
1 stars 0 forks source link

f-string parser: Fix error messages in `test_unterminated_string` #163

Open pablogsal opened 3 years ago

pablogsal commented 3 years ago

The error messages in test_unterminated_string need updating:

FAIL: test_unterminated_string (test.test_fstring.TestCase) (str='f\'{("x}\'')\n----------------------------------------------------------------------
  File "<string>", line 1
    f'{("x}'
        ^
SyntaxError: unterminated string literal (detected at line 1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pablogsal/github/python/f-string-grammar/Lib/test/test_fstring.py", line 32, in assertAllRaise
    with self.assertRaisesRegex(exception_type, regex):
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: "f-string: unterminated string" does not match "unterminated string literal (detected at line 1) (<string>, line 1)"

There is a bunch of them that need also updating in test_missing_expression but we need to check first if we need to change the error we emit in the tokenizer

FFY00 commented 3 years ago

Old:

$ ./python -c 'f\'{"x\''
  File "<string>", line 1
    f'{"x'
          ^
SyntaxError: f-string: unterminated string

New:

$ ./python -c 'f\'{"x\''
  File "<string>", line 1
    f'{"x'
       ^
SyntaxError: unterminated string literal (detected at line 1)

The error seems good, but I wonder if we should still prefix it with f-string: prefix like previously.

pablogsal commented 3 years ago

I think it makes sense, I think it should be more or less straightforward because we know if we are in fstring mode.

FFY00 commented 3 years ago

This is the same in all other errors AFAICT, so I was just thinking of adding the prefix to the error string after parsing, but I am not sure where that code is. That so far has been my biggest problem so far, knowing the code structure and what each bit does :sweat_smile:

isidentical commented 3 years ago

so I was just thinking of adding the prefix to the error string after parsing but I am not sure where that code is

For the stuff that is raised at the tokenizer (e.g unterminated string literals), they all should go through this IIRC. https://github.com/we-like-parsers/cpython/blob/435fead4bc7e6a5103bfcfe7ebb67e08ff7dcd04/Parser/tokenizer.c#L1101

For the parser this might help, https://github.com/we-like-parsers/cpython/blob/435fead4bc7e6a5103bfcfe7ebb67e08ff7dcd04/Parser/pegen.c#L366