search

gradle / gradle

Adaptable, fast automation for all
https://gradle.org
Apache License 2.0
16.64k stars 4.66k forks source link

Antlr plugin cannot find tokens file when grammar is separated and a package is used #29946

Open Dwarfley opened 1 month ago

Dwarfley commented 1 month ago

Current Behavior

My project uses the antlr gradle plugin. The grammar are stored in separate lexer/parser files with the parser grammar using the tokenVocab option to locate the tokens. These grammar files are located in a package folder structure such as src/main/antlr/org/example/antlr/. When building the project it fails with the following error message.

error(114): org\example\antlr\TestParser.g4:5:14: cannot find tokens file org\example\antlr\TestLexer.tokens
warning(125): org\example\antlr\TestParser.g4:13:3: implicit definition of token Test in parser

Expected Behavior

I expected the plugin to find the tokens generated by the lexer grammar and not fail with an error.

Context (optional)

The plugin works when both lexer and parser grammar is placed in the same file. The separate grammar files do not. I don't know if this is supposed to work but prevented by a bug or simply a limitation of the plugin. The documentation for the antlr plugin doesn't specify either case. Having the option of separating lexer/parser grammar would be helpful when dealing with complex grammar. If this is a limitation of the plugin it should perhaps be stated in the docs.

Steps to Reproduce

A minimal reproducible example containing test grammar in separate files. The package used is org.example.antlr. No java code is provided as the problem relates to the antlr plugin. The wrapper version is also 8.9.

test.zip

Gradle version

8.9

Build scan URL (optional)

No response

Your Environment (optional)

Antlr version: 4.13.1 Java version: 21.0.1 2023-10-17 LTS OS: Windows 10

alllex commented 1 month ago

This issue needs a decision from the team responsible for that area. They have been informed. Response time may vary.

A potentially related issue:

octylFractal commented 1 month ago

This is a valid issue, and I believe the same as #24452, however I'm not sure if the problem is in the Gradle plugin or ANTLR itself. We're not looking at this area currently, so there is no estimate on when we will investigate it.