search

lark-parser / lark_cython

Cython plugin for Lark, reimplementing the LALR parser & lexer for better performance
MIT License
45 stars 5 forks source link

fix UnexpectedToken handling exception #12

Closed ernestoarbitrio closed 1 year ago

ernestoarbitrio commented 1 year ago

I don't know if this is the correct solution but it fixes the case described below

👇🏽

with a grammar like:

display_decimals: "DECIMALS"i WS INT
start: display_decimals
%import common (INT, WS)

and a script like

script = """
      bla
"""

It raises

*** AttributeError: 'lark_cython.lark_cython.LexerThread' object has no attribute '_Token'

This PR handle the right exception when the token is wrong rather than raising an Attribute error.

ernestoarbitrio commented 1 year ago

Hi @erezsh sorry, but I was wondering if there's any chance I can get a code review here?

ernestoarbitrio commented 1 year ago

Ok it seems that you have covered this on the latest PR you've merged. Should I delete this or we can keep at least the test?

erezsh commented 1 year ago

Sorry that I didn't attend to this earlier.

If you think the test is still valuable, I don't mind adding it. But I don't like testing for specific errors strings, since they can change for a variety of good reasons.

ernestoarbitrio commented 1 year ago

I will close it then