Investigate running tests through maven

[chrisparnin]

Let's generate a new goal, "randomtest", that is in maven pom.xml.

Basically, it runs your new tests. Hopefully, properly loads all the dependencies that it needs.

[rohancme]

Creating new goals/phases is probably not the best approach at the moment. For one, it's a decent amount of extra effort. @chrisparnin How important do you think it is to have a separate goal for the newly generated tests?

For another it seems like behavior is different in Maven 2 vs 3 which is not something I think I'd want to tackle at this point in the project.

Stack Overflow answers discussing this in detail

Instead, I think it might be more useful to work on automating this workflow which has been successful on the projects I've tried so far. The end result is that the tests are run as part of the maven test phase.

Steps that need to be automated:

1. Go into [repo_dir]
2. mvn assembly:assembly -DdescriptorId=jar-with-dependencies

3. For each class:

   1. java -classpath randoop-.jar: [repo_dir]/target/-jar-with-dependencies.jar randoop.main.Main gentests --testclass=className
   2. Modify RegressionTest.java to add an import statement and add a main method:

   import junit.framework.JUnit4TestAdapter;
   
   public class RegressionTest{ 
   
      public static void main(String[] args){
          junit.textui.TestRunner.run (suite());
      }
   
      // The suite() method is helpful when using JUnit 3 Test Runners or Ant.
      public static junit.framework.Test suite() 
      {
         return new JUnit4TestAdapter(RegressionTest.class);
      }
   
   }

   1. Repeat for RandoopTest*
   2. cp RegressionTest*.java into test/origFilePath/
4. mvn clean test

Bonus: I can easily compare PITest coverage before and after adding the new tests.

[rohancme]

Also, that particular modification to the RegressionTest class takes of a problem I face when a repo uses a version of Junit <4 because Junit 3 has a different way of identifying tests.

[chrisparnin]

So for now, avoiding running an additional mavengoal. In future can investigate how to make plugin so we can introduce new lifecycle. Easier to just merge into existing junit tests.