Extend testrunner to be gradle compatible

byjiang1996 commented 3 years ago

Hi, I'm Binyao Jiang attending CS527 this semester and my project is to extend idflakies and testrunner to be gradle compatible.

This PR depends on this testrunner open PR: https://github.com/TestingResearchIllinois/testrunner/pull/16 and another iDFlakies open pr: https://github.com/idflakies/iDFlakies/pull/28.

Following is the explanation for the PR commits: 1st commit: Rebase the codebase to the open PR: https://github.com/idflakies/iDFlakies/pull/28 since we have to modify multiple same files simultaneously. 2nd commit: Update when Testrunner moves core plugin to coreplugin module. 3rd commit: Update when Testrunner wraps MavenProject object into ProjectWrapper. After this step, iDFlakies's interface will be maven/gradle irrelevant so that we can use the same iDFlakies code to generate iDFlakies maven plugin and gradle plugin.

winglam commented 3 years ago

I am not too familiar with Gradle. Can you provide some instructions for how one should modify a gradle project to use testrunner/iDFlakies and the command to run iDFlakies? (similar to what is on the current quickstart guide)

ericssy commented 3 years ago

Hi Wing, here is a brief explanation on how one should modify a gradle project for iDFlakies/testrunner. Do you think it's ok to include it with this PR?

Using iDFlakies on a Gradle project

Install iDFlakies and testrunner

Go to the testrunner root directory, and run:

mvn clean install

Then go to iDFlakies root directory, and run:

mvn clean install

Automatically setting up the build.gradle for iDFlakies

Run the following command to automatically setup the build.gradle for iDFlakies.

bash gradle_modify/modify_gradle.sh path_to_gradle_project

Manually setting up the build.gradle for iDFlakies

We can configure the build.gradle file manually instead. We need to follow the rule listed below to successfully set up the project.

First we need to add the following dependencies and repository to the buildscript{} in build.gradle

buildscript {
  repositories {
     ...
     mavenLocal()
     ...
  }

  dependencies {
     ...
     classpath group: 'edu.illinois.cs',
     name: 'idflakies', 
     version: '1.1.0-SNAPSHOT'

     classpath group: 'edu.illinois.cs',
     name: 'testrunner-gradle-plugin', 
     version: '1.2-SNAPSHOT'
     ...
  }
}

Then, add apply plugin: 'testrunner' to the build.gradle file. The location of this code snippet depends on whether build.gradle file has allprojects{} and subprojects{}.
- If there are both allprojects{} and subprojects{} methods in gradle.build, then the apply plugin: "testrunner" command should be added to allprojects{} only.
- If only allprojects{}method is in gradle.build, the command is added to allprojects{}
- If only subprojects{} method is in gradle.build, the command is added to both subprojects{} and outside of all the methods in the build file
- If there's no allprojects{} and subprojects{} in gradle.build, only add the command outside of all the methods
Here is an example where this code snippet is added outside of all methods.
```
buildscript {
  repositories {
     ...
     mavenLocal()
     ...
  }

  dependencies {
     ...
     classpath group: 'edu.illinois.cs',
     name: 'idflakies', 
     version: '1.1.0-SNAPSHOT'

     classpath group: 'edu.illinois.cs',
     name: 'testrunner-gradle-plugin', 
     version: '1.2-SNAPSHOT'
     ...
  }
}

apply plugin: 'testrunner'
```

Running iDFlakies on Gradle project

After the build.gralde has been configured, one can run iDFlakies on the Gradle project with the following command:

./gradlew testplugin -Dtestplugin.className=edu.illinois.cs.dt.tools.detection.DetectorPlugin -Ddetector.detector_type=random-class-method -Ddt.randomize.rounds=10 -Ddt.detector.original_order.all_must_pass=false

winglam commented 3 years ago

Thanks for the instructions to use iDFlakies on a gradle project.

Can you clarify where the gradle_modify/modify_gradle.sh is? I did not find this file in this PR or the testrunner PR

Would this modify_gradle.sh script automatically handle all of the cases related to allprojects{} and subprojects{}?

ericssy commented 3 years ago

The gradle_modify/modify_gradle.sh wasn't included in these two PR. Do yo want me to include it in the iDflakies PR?

Yes, the modify_gradle.sh would automatically handle all the cases according to the rule listed above.

byjiang1996 commented 3 years ago

Hi Wing, apart from the gradle modify issue, do you have any suggestions for the remaining part? The code revision or test could be done in parallel.

byjiang1996 commented 3 years ago

Fix typo above.
Merge conflict is resolved.
Idflakies is synced with testrunner in another PR.

UT-SE-Research / iDFlakies