antlr / antlr4

ANTLR (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing, or translating structured text or binary files.
http://antlr.org
BSD 3-Clause "New" or "Revised" License
16.97k stars 3.26k forks source link

Missing error messages in C# #2056

Open dbushuev opened 6 years ago

dbushuev commented 6 years ago

I ran my tests in C# environment with 2 small grammars

1) Here is my first grammar:

grammar Request;

numbers : NUMBER+ ;

NUMBER : [0-9]+ ;

WS : [ \r\n\t] + -> skip ;

Here is the test input: 3 4 5 X 6 7 8 When I run parser I get error (as expected): line 1:6 token recognition error at: 'X'

2) Now I add 2 rules to recognize identifiers (second grammar):

grammar Request;

numbers : NUMBER+ ;
identifiers : ID+ ; // <- new parser rule

NUMBER : [0-9]+ ;
ID : [_A-Za-z] [_0-9A-Za-z]* ; // <- new lexer rule

WS : [ \r\n\t] + -> skip ;

And run it again with same starting rule (numbers) and same input. The result - NO ERRORS.

I have tested both grammars in java test rig and got errors both times. However in C# environment second grammar does not produce any errors.

Just in case, here is my C# code

ICharStream stream = new AntlrInputStream("3 4 5 X 6 7 8");
RequestLexer lexer = new RequestLexer(stream);
var tokens = new CommonTokenStream(lexer);
RequestParser parser = new RequestParser(tokens) { BuildParseTree = true };
IParseTree tree = parser.numbers();

Thanks

KvanTTT commented 6 years ago

Try to add EOF token at the end of numbers rule and try again. You should get the parse error.

ericvergnaud commented 6 years ago

Is that with the official C# runtime or the one from tunnel vision ?

Envoyé de mon iPhone

Le 14 oct. 2017 à 03:31, Dimitry Bushuev notifications@github.com a écrit :

I ran my tests in C# environment with 2 small grammars

Here is my first grammar: grammar Request;

numbers : NUMBER+ ;

NUMBER : [0-9]+ ;

WS : [ \r\n\t] + -> skip ; Here is the test input: 3 4 5 X 6 7 8 When I run parser I get error (as expected): line 1:6 token recognition error at: 'X'

Now I add 2 rules to recognize identifiers (second grammar): grammar Request;

numbers : NUMBER+ ; identifiers : ID+ ; // <- new parser rule

NUMBER : [0-9]+ ; ID : [_A-Za-z] [_0-9A-Za-z]* ; // <- new lexer rule

WS : [ \r\n\t] + -> skip ; And run it again with same starting rule (numbers) and same input. The result - NO ERRORS.

I have tested both grammars in java test rig and got errors both times. However in C# environment second grammar does not produce any errors.

Just in case, here is my C# code

ICharStream stream = new AntlrInputStream("3 4 5 X 6 7 8"); RequestLexer lexer = new RequestLexer(stream); var tokens = new CommonTokenStream(lexer); RequestParser parser = new RequestParser(tokens) { BuildParseTree = true }; IParseTree tree = parser.numbers(); Thanks

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

sharwell commented 6 years ago

@ericvergnaud The input stream type in my fork is ANTLRInputStream, so assuming the example is accurate then this would would the standard runtime.

dbushuev commented 6 years ago

Yes, official