1 create command line testing tool

[Groostav]

This is a PR for #1, empoweroperations/OASIS#708

This is the first commit for this new repo. It contains the build system and initial implementation in kotlin for singularity.exe.

It contains:

• a build script (that requires exe4j!) that generates an executable
• a run configuration so you can simply run it from intelliJ
• a testing configuration that runs the unit tests from intelliJ

Check out the branch, and poke around at these run configs. I want to make sure they work on other peoples machines, because a lot of my time was spent avoiding dependencies on hard-coded paths.

This build system uses exe4j, which I haven't properly configured to download and install through gradle, you need to have it installed before hand. Similarly, I was hoping to do some clever publishing of artifacts to maven or a local maven repo or something, but ultimately I'm just copy-pasting exe files around (read; copying the singularity.exe file from the build output here into /.build in OASIS).

Usages:

Sample output from singularity.exe --help:

Usage: <main class> [options]
  Options:
    -e, --exit
      executable behaiviour distribution, repeated once for each possible
      outcome eg '-e 10% code:2 -e 30% hang'
      Default: [--exit 100.0% code:0]
    -g, --generator
      generator function to create result values
      Default: RosenbrockAdaptiveN
    -h, --help
      Prints a description and this usage summary
      Default: false
    -i, --input
      path read input values from
      Default: .\input.properties
    -o, --output
      path to write simulation result values to
      Default: .\output.properties
    -t, --time
      time to execute for in millis
      Default: 200

Thus, if you wanted to get the tool to run instantly, rosenbrock 10, with hard-coded checks against the names (ie "x1" through "x10"), you can do this:

singularity.exe --generator RosenbrockX1ThroughX10 --time 0

This would use the following defaults:

default	value	description
input file	./input.properties	causes singularity to attempt to read from workingDir/input.properties for key-value pairs in the form x1=1.2
output file	./output.properties	singularity will put its generators output in this file. Right now only rosenbrock is supported, so it would put the result f1=1.234 in workingDir/output.properties
exit type & code	--exit 100% code:0	causes singularity to exit with exit-code 0 always

We can change the return type to something like exit with exit-code 7 for 5% of the time like this:

singularity.exe --generator RosenbrockX1ThroughX10 --exit 95% code:0 --exit 5% code:7

And so on. I'm hoping its fairly descriptive.

from the commit message:

[Issue #1] Create command line testing tool

• this is a surrogate commit/issue for empoweropations/OASIS#708
• this commit represents a skeletal implementation of a command line tool for use as a stand-in for simulation programs like MATLAB and ANSYS.
• theres a reasonably good help message, but at this point most of the functionality is around which files you want to modify, and how long the tool should take to execute
• no follow-up issues yet, but if you read the source, there's a lot of obvious directions we could go

[blui]

October 31, 2016

Looking over this pull request.

• Only 2 generators at the moment; RosenbrockX1ThroughX10 and RosenbrockAdaptiveN
  • So what I should do is test variations of these + exit values, etc.

Basic test cases:

• [x] Getting this to work with basic configuration; --generator RosenbrockX1ThroughX10 --time 0
  • I adjusted the values in input.properties and output.properties and after running this, I do see these values being changed.
• [x] Getting this to work with basic configuration; --generator RosenbrockX1ThroughX10 --exit 95% code:0 --exit 5% code:7
  • I adjusted the values in input.properties and output.properties and after running this, I do see these values being changed.

Random test cases:

• [x] Testing 100% exit with exit code 1
• [x] Testing 50% exit with exit code 1
  • Okay, so behaviours have to sum up to 100%. Good catch.
• [x] Testing 50% exit with exit code 1 and 50% exit with exit code 2
  • Nice. So I ran this a few times and I do see it exiting sometimes with code 2 and sometimes with code 1.
• [x] Testing 25% exit with exit code 1, 25% exit with exit code 2, and 50% exit with exit code 3

Issue-specific test cases:

• [x] Singularity with OASIS without using any specific conditions (everything is default)
  • So this worked with OASIS via IntelliJ and with the singularity.config file being empty.
  • This worked with OASIS installed and with the singularity.config file being empty.
• [x] Singularity with OASIS; using a specified generator 'RosenbrockX1ThroughX10'
  • This worked with OASIS via IntelliJ and with the singularity.config file being empty. The command option was specified within OASIS.
  • This worked with OASIS installed and with the singularity.config file being empty. The command option was specified within OASIS.
• [ ] Singularity with OASIS; using a specified exit code 'exit code: 2' - I'm having a bit of difficulty specifying this option. In OASIS in external tool, if I set '-e' as the option and '100% code:2' as the argument, then I get system.out.println saying it's expecting 2 values after the option -e. If I set '-e 100% code:2' entirely in the options, then I can't start the optimization.
• [x] Singularity with OASIS; using a specified time out value '400 ms'
  • This worked with OASIS via IntelliJ and with the 'singularity.config' file being empty. The command option was specified within OASIS.
  • This worked with OASIS installed and with the singularity.config file being empty. The command option was specified within OASIS.
• [x] Singularity with OASIS; using a specified input file and output file
  • There's no need to test this for use within OASIS because input and output files are already specified as part of the external tool setup.

[blui]

November 01, 2016

Confirmed with Geoff that the exit command option should be entered as:

-e // 100% // code:2 (split into 3 parts)

• [x] Singularity with OASIS; using a specified exit code (code:2)
  • Without catching exit code 2 we see the error message being thrown in OASIS indicating that the optimization is running into an unknown exit code 2. This is proper behaviour.
  • Specified exit code 2 in the external tool settings and now seeing OASIS working properly.