lark-parser / lark

Lark is a parsing toolkit for Python, built with a focus on ergonomics, performance and modularity.
MIT License
4.64k stars 397 forks source link

However, trying to define a string with quotes instead of double quotes doesn't work #1332

Closed Fxztam closed 10 months ago

Fxztam commented 10 months ago

Hi,

I'm trying to use LARK as a search pattern engine for the given code and have to redefine a string with Quotes "'".

I'm using parser=lalr and lexer=contextual but I am not able to create a quoted separated string containing all rules to be ignored;

although the string is recognized, it is parsed, and a violation of a rule is recognized.

How can I define a quoted string so that all rules contained in it are ignored?

Thanks, Fried

erezsh commented 10 months ago

Sorry, but I don't understand at all what you're asking. Try to include a code example.

Fxztam commented 10 months ago

If I understand correctly, a string terminal in Lark is defined with double quotes. However, my Lark-built code handler for a given grammar requires the string as a quote definition; any characters and rules should be ignored in '....bla.bla...rule(1)..rule(n)..' and i.e. everything must be ignored in quotes-separated strings (no lexing & parsing).

I am going to build an extracted example for this where in my string definition Lark parses the string content.

Thanks, Fried

Fxztam commented 10 months ago

I have in my example to use the newly defined quoted-string '. any char or any rules ..'
to ignore the included content means no parsing of the quoted string e.g.

see in the example line 59 :

x:= '#+*()/&!"§$%&/()=? select a into ' ;

that brings me to:

` raise UnexpectedToken(token, expected, state=self, interactive_parser=None) lark.exceptions.UnexpectedToken: Unexpected token Token('INTO', 'into') at line 4, column 54. Expected one of:

How can I define the quoted string so that its content is not parsed?

Thanks, Fried

erezsh commented 10 months ago

Look at how strings are defined in the python grammar: https://github.com/lark-parser/lark/blob/master/lark/grammars/python.lark#L284C26-L284C26

Any reason that you can't use this approach?

Fxztam commented 10 months ago

Yes, of course, the string definition in the Python grammar works very well!

Thank you very much, Fried

erezsh commented 10 months ago

Great!