Maven and Eclipse plugins: option to put generated files in same dir as .g4

[ccleve]

It would be very helpful to have an option in the maven plugin (and the Eclipse plugin) to put the generated files in the same directory as the .g4 file. The reasons are:

M2Eclipse, the maven-2-eclipse plugin, does not correctly add /target/generated-sources as a source dir, so nothing gets compiled. You need to do manual fixup to make it work.

The maven-source-plugin doesn't attach sources in the /target dir, which means that source attachments go out without the generated files. You can no longer step through the code.

Version control systems like git ignore the /target directory, which means that generated files don't get versioned. No way to go back and see what happened.

Bottom line: using a non-standard directory structure generates problems.

Fortunately, putting the generated files in the same dir as the .g4 file solves all of the problems automatically. Plus it keeps everything together in one package, which is much cleaner.

No need to change the default scheme for people who prefer it the way it is now. Just add an option to do it differently.

[sharwell]

You can add <outputDirectory> to the configuration of the antlr4-maven-plugin to set the location where source files are placed. By default they are placed in ${project.build.directory}/generated-sources/antlr4, but you could set that to the same location as you have <sourceDirectory> if you want the generated files placed next to the grammars.

M2Eclipse, the maven-2-eclipse plugin, does not correctly add /target/generated-sources as a source dir, so nothing gets compiled. You need to do manual fixup to make it work.

This is likely a shortcoming of the implementation. Currently the antlr4-maven-plugin uses a BuildContext to create files, but it doesn't call buildContext.refresh. I'm not sure what, if any, impact that would have on the Eclipse workspace.

:bulb: It would make sense to create a new issue requesting a solution to this nuance.

---

I believe the limitation/bug encountered is the main source of all the other points you listed. I'll address each of the main ones here (keep in mind much of this is only my opinion).

The maven-source-plugin doesn't attach sources in the /target dir, which means that source attachments go out without the generated files. You can no longer step through the code.

I can't reproduce this. I checked multiple distributions of ANTLR 4 and StringTemplate which are built using different versions of Java, Maven, ANTLR, etc., and in all cases the sources jar produced for the artifacts included the generated grammar files which existed only in the target folder at the time of the build.

Version control systems like git ignore the /target directory, which means that generated files don't get versioned. No way to go back and see what happened.

This is by design. The generated code is a function of the input grammar and the version of ANTLR, both of which are versioned. To see the code associated with a particular release, it's best to reference the sources jar which was created at the time of release. To generate the code for a particular commit, simply check out that commit and run the generate-sources target.

Remember that subtle changes in a grammar can cause cascading changes throughout a very large generated parse file. Including the generated code in source control is a sure way to rapidly increase the size of your repository with data that is trivially reproduced during a build.

Bottom line: using a non-standard directory structure generates problems.

The antlr4-maven-plugin has always defaulted to a standard Maven directory structure layout. Recent releases of the plugin now go out of their way to accommodate Eclipse's non-standard use of Maven, but it seems not all of the problems caused by Eclipse have been addressed to date.

Fortunately, putting the generated files in the same dir as the .g4 file solves all of the problems automatically. Plus it keeps everything together in one package, which is much cleaner.

I recommend not putting the generated files in the same directory as the grammar files.

[sharwell]

Let me know if setting <outputDirectory> works for you and I'll close this issue. :+1:

[ccleve]

I tried and and the system didn't find the .g4 file to parse. Not sure if it's a bug or my error.

But more generally, this solution doesn't work if your app has more than one grammar and you want to keep the generated files together with the grammar.

There's no downside to adding an option to keep the files in one directory. You don't have to use it if you don't want to.

And I disagree with you about the generated parsers. Disk space is not an issue, usually, and if it is, just don't use the option.

[parrt]

a new pom.xml will be forthcoming.

[sharwell]

@ccleve I missed your reply previously. Is your project open source by any chance, where I could test this out locally?

[ccleve]

@sharwell Unfortunately not, it's just internal test code. It's nothing complicated, though. Plain-vanilla .g4 file. The plugin config is below. The whole project isn't compiling now so it's hard for me to give you an error message, but I'll try later.

            <!-- Plugin to compile the g4 files ahead of the java files See https://github.com/antlr/antlr4/blob/master/antlr4-maven-plugin/src/site/apt/examples/simple.apt.vm 
                Except that the grammar does not need to contain the package declaration 
                as stated in the documentation (I do not know why) To use this plugin, type: 
                mvn antlr4:antlr4 In any case, Maven will invoke this plugin before the Java 
                source is compiled -->
            <plugin>
                <groupId>org.antlr</groupId>
                <artifactId>antlr4-maven-plugin</artifactId>
                <version>4.5</version>
                <executions>
                    <execution>
                        <configuration>
                            <goals>
                                <goal>antlr4</goal>
                            </goals>
                            <sourceDirectory>${basedir}/src/main/com/dieselpoint/antlr</sourceDirectory>
                            <outputDirectory>${basedir}/src/main/com/dieselpoint/antlr</outputDirectory>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

[emculturate]

I am experiencing a related issue now with Eclipse Neon 2 Release and ANTLR4 plugin version 4.5. Here is my pom fragment `

org.antlr

    <artifactId>antlr4-maven-plugin</artifactId>
    <executions>
       <execution>
         <configuration>
           <goals>
             <goal>antlr4</goal>
           </goals>
           <outputDirectory>${basedir}/src/main/java</outputDirectory>
         </configuration>
       </execution>
     </executions>
  </plugin>`

What is happening is that the grammars are being properly generated but they are not being moved into place in the java source directory. They are staying in the /target/generated-sources/antlr4 subdirectories.

I had this working in an Eclipse MARS environment using 4.4, but needed to upgrade. Let me know if I should create a new issue?

[parrt]

try 4.6?

[jimidle]

This is an Eclipse config thing really.

You need to add the target/generated-sources directory as a source directory in your product configuration so that Eclipse knows about it. after upgrading to Luna, I gave up and switched to IntelliJ. I have not had occasion to regret it.

However, generally, you should not move things from their default Maven place, as Maven expects it - it usually ends in tears ;) If you leave things in the generated-sources directory, then everything should build without changing any configurations.

Jim

On Thu, Jan 19, 2017 at 4:30 AM, emculturate notifications@github.com wrote:

I am experiencing a related issue now with Eclipse Neon 2 Release and ANTLR4 plugin version 4.5. Here is my pom fragment

org.antlr antlr4-maven-plugin antlr4 ${basedir}/src/main/java

What is happening is that the grammars are being properly generated but they are not being moved into place in the java source directory. They are staying in the /target/generated-sources/antlr4 subdirectories.

I had this working in an Eclipse MARS environment using 4.4, but needed to upgrade. Let me know if I should create a new issue?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/antlr/antlr4/issues/831#issuecomment-273592494, or mute the thread https://github.com/notifications/unsubscribe-auth/ABP5sPNJ_x5kwBxkJvmycQofewg8B2-8ks5rTnZagaJpZM4DrXLs .