search

EvoSuite / evosuite

EvoSuite - automated generation of JUnit test suites for Java classes
http://www.evosuite.org
GNU Lesser General Public License v3.0
841 stars 342 forks source link

Experiment runner setup with Docker #326

Closed mitchellolsthoorn closed 3 years ago

mitchellolsthoorn commented 4 years ago

The current Docker images only allow one instance of EvoSuite to be run. To run large-scale experiments, multiple parallel instances of EvoSuite are desired.

I have created an experiment_runner.sh script that provides a nice interface for this. This script allows for highly configurable experiments without any code changes. I have added two new Dockerfiles to make it easy to run the experiment on a local computer or a cluster.

I have also updated the README.md with extra documentation for this functionality

mitchellolsthoorn commented 4 years ago

I also have an R script that can automatically extract the results from this experiment runner and give the statistical significance, the effect size, and some graphs for an overview

mitchellolsthoorn commented 4 years ago

I added some extra functionality to the script and docker image to allow for better seeds for the random number generator. Before running all the different configurations of the experiment it creates a file called SEEDS containing random seeds (from /dev/urandom) for the different executions of the experiment. This file can be used to replicate the experiment later by putting it back

gofraser commented 3 years ago

This looks fine, is there a reason this is marked as work in progress?

mitchellolsthoorn commented 3 years ago

@gofraser No particular reason, I forgot to mark it ready for review. I also added the ability to manually override the classpath since some members of our group found that useful/necessary for some experiments. I cloned the branch from scratch and tested if everything still works as it is supposed to, just as a safety check.

gofraser commented 3 years ago

Awesome, thanks!