search

melix / antlr4-gradle-plugin

Antlr4 plugin for Gradle
Apache License 2.0
41 stars 13 forks source link

Question: adding Antlr4TestTask #4

Closed stephengaito closed 10 years ago

stephengaito commented 10 years ago

Cédric,

I would like to add a task which runs the ANTLR4 TestRig, to enable regression integration testing of the grammars being developed.

Should I work this into a pull request for this project (my preference), or should I make this a separate gradle plugin which depends upon antlr4-gradle-plugin?

Regards, Stephen Gaito

stephengaito commented 10 years ago

Cédric,

After some more thought about the whole problem, I have decided that I should simply create a separate antlr4-gradle-regressionTestPlugin project.

The ANTLR4 TestRig is not really sufficient for automated regression testing of lots of test documents, and the required changes are really not the subject of a "simple" pull request on the ANTLR4 project (which I have done in the past).

Hence the antlr4-gradle-regressionTestPlugin will be both a gradle plugin as well as the code for a regressionTestRig for ANTLR.

SO, unless you really think I should merge my work into your antlr4-gradle-plugin, I will consider this "issue" closed.

Again, many thanks for such a simple and clean gradle plugin as an open source project. Your gradle plugin provides a very useful example as well as tool.

Regards, Stephen Gaito

melix commented 10 years ago

Would be good to have it as a task, yes!

stephengaito commented 10 years ago

Ok, let me flesh out my solution and then we can have a look at integration.

My main concern is that the ANTLR license is 3Clause BSD. The BSD license should work nicely enough with Apache v2 -- but I need to think carefully how to do this since the bulk of the code will be derived from ANTLR's TestRig and Trees classes.

adobni commented 10 years ago

I would also like the original ANTLR4 TestRig task. It seems useful for unit testing grammars even if it is not good enough for regression tests. Regards Alejandro

stephengaito commented 10 years ago

Cédric,

I have now opened up two GitHub repositories https://github.com/fandianpf/antlr4-gradle-regressionTestPlugin and https://github.com/fandianpf/antlr4-regressionTestRig

The antlr4-regressionTestRig repository is a simple extension of the ANTLR4 runtime's TestRig tool. My extensions have been primarily to ensure as much output as possible is captured in a file (as opposed to the System.out). There are a few other extensions, see the project readme for details. At some point, I would consider attempting to get my extensions worked into the original ANTLR4 runtime TestRig tool... all depending upon my use, workload and Terence Parr's receptiveness to such changes.

The antlr4-gradle-regressionTestPlugin is a simple gradle plugin, based on your plugin, which runs the antlr4-regressionTestRig against a FileTree collection of example documents. At this point the antlr4test task does not fail ... I will have to think about how to integrate this into the overall process.

The antlr4-gradle-regressionTestPlugin repository is meant as a short term experiment while I/we understand what changes might be useful to either the RegressionTestRig or the antlr4test task. Once we feel there are no further obvious changes, I would happily create the required pull request to integrate my antlr4test task into your repository and package paths.

Please have a look at both projects, and if appropriate, see how easy (or not) the new plugin is to use in your processes. Any comments will be happily received as issues on either repository.

stephengaito commented 10 years ago

adobni,

Given the overlap between the TestRig and my RegressionTestRig tools, I do not see the point of (or I do not have the time/resources for) providing two antlr4test tasks.

Please have a look at the readme for the antlr4-regressionTestRig project: https://github.com/fandianpf/antlr4-regressionTestRig and see if the extended regressionTestRig is good enough for your needs (I think it will be).

At the moment the antlr4test task is in the separate gradle plugin: https://github.com/fandianpf/antlr4-gradle-regressionTestPlugin As I said in my reply to Cédric, I fully intend to integrate my antlr4test task into his antlr-gradle-plugin, once things have stablized.

Your comments will be happily received as issues on either project.

adobni commented 10 years ago

Thanks Stephen, I'll try as soon as I can. To be clear I only proposed keeping the original issue open because I thought that it would be simpler but from the readmes I can see is an extended version of TestRig which is fine for me. I want to thank you all for creating these plugins and share them.

Regards Alejandro

On Thu, Jun 12, 2014 at 8:52 AM, stephengaito notifications@github.com wrote:

adobni,

Given the overlap between the TestRig and my RegressionTestRig tools, I do not see the point of (or I do not have the time/resources for) providing two antlr4test tasks.

Please have a look at the readme for the antlr4-regressionTestRig project: https://github.com/fandianpf/antlr4-regressionTestRig and see if the extended regressionTestRig is good enough for your needs (I think it will be).

At the moment the antlr4test task is in the separate gradle plugin: https://github.com/fandianpf/antlr4-gradle-regressionTestPlugin As I said in my reply to Cédric, I fully intend to integrate my antlr4test task into his antlr-gradle-plugin, once things have stablized.

Your comments will be happily received as issues on either project.

— Reply to this email directly or view it on GitHub https://github.com/melix/antlr4-gradle-plugin/issues/4#issuecomment-45882160 .

stephengaito commented 10 years ago

Cédric and Alejandro,

I have just released version 0.2 of both antlr4-regressionTestRig and antlr4-gradle-regressionTestPlugin. This version significantly improves the numbers of metrics captured about the parse of each document. It also duplicates the directory structure of the test documents into the results output directory.

If and when you happen to try these out, let me know what you think.

Regards, Stephen Gaito

adobni commented 10 years ago

Stephen: I got this error when I tried the plugin: gradle build

FAILURE: Build failed with an exception.

BUILD FAILED

The same if I try changing the version to 0.2.

I attached my build.gradle file.

On Fri, Jun 13, 2014 at 12:52 PM, stephengaito notifications@github.com wrote:

Cédric and Alejandro,

I have just released version 0.2 of both antlr4-regressionTestRig and antlr4-gradle-regressionTestPlugin. This version significantly improves the numbers of metrics captured about the parse of each document. It also duplicates the directory structure of the test documents into the results output directory.

If and when you happen to try these out, let me know what you think.

Regards, Stephen Gaito

— Reply to this email directly or view it on GitHub https://github.com/melix/antlr4-gradle-plugin/issues/4#issuecomment-46027801 .

stephengaito commented 10 years ago

Alejandro,

Alas this is a bit of a learning curve for me. I had neglected to update the antlr4-regressionTestRig version in the Antlr4TestPlugin file. This presented an error for me, though not one like you seem to be having.

I also notice a small typo in the installation example in the installation section of the README.md file ( https://github.com/fandianpf/antlr4-gradle-regressionTestPlugin#installation ) (I used a " when I should have used a ' ) It is possible that this typo if you had directly copied and pasted it into your build.gradle script, was confusing Gradle.

Alas I have not been able to see your build.gradle file, so I can not provide any further comments.

Regards, Stephen Gaito