search

renatahodovan / grammarinator

ANTLR v4 grammar-based test generator
Other
340 stars 61 forks source link

Error in processor.py when processing PartiQL grammars #212

Closed miryung closed 6 months ago

miryung commented 7 months ago

I am trying to process PartiQL ANLTR grammar located here: https://partiql.org/syntax/antlr.html

I attached g4 grammars with txt extension (to allow upload). I checked that these grammars do not have syntax errors and are able to parse input files correctly. PartiQL.g4.txt PartiQLTokens.g4.txt

miryung@bcd074185726 ANTLRGrammar % grammarinator-process --no-actions -o PartiQL-example/fuzzer PartiQL.g4 PartiQLTokens.g4 Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.11/bin/grammarinator-process", line 8, in sys.exit(execute()) ^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grammarinator/process.py", line 67, in execute ProcessorTool(args.language, args.out).process(args.grammar, options=options, default_rule=args.rule, encoding=args.encoding, errors=args.encoding_errors, lib_dir=args.lib, actions=args.actions, pep8=args.pep8) File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grammarinator/tool/processor.py", line 444, in process self._analyze_graph(graph) File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grammarinator/tool/processor.py", line 906, in _analyze_graph for out_v in graph.vertices[v].out_neighbours:


KeyError: None[](url)
renatahodovan commented 7 months ago

Hi @miryung !

Thanks for the report. The problem with the provided grammar is that it consists of a combined grammar (PartiQL.g4) and a lexer grammar (PartiQLTokens.g4), instead of using either a pair of parser & lexer grammars or a single combined grammar as described in the ANTLR docs.

ANTLR will generate a single parser only from your combined grammar since it doesn't contain any lexer rules and a lexer from your lexer grammar. So, those two can work together as a parser-lexer pair. However, this is a bit of an edge case. It should be safer to define PartiQL.g4 explicitly as a parser grammar. Not to mention that Grammarinator does not support this edge case.

However, by adding the parser keyword to the first grammar definition line of PartiQL.g4 (making it parser grammar PartiQL;), Grammarinator can also handle the two grammar files.

Please, tell me if it worked for you!

miryung commented 6 months ago

It worked for me. Thanks Renata.