Make it easier to test various configurations

[ronawho]

Testing the 'std' configuration is relatively straightforward, but there's no prescribed way to test other configurations like gasnet, numa, valgrind, etc.

I personally have a lame ~/.setChplHelpers.sh file with various helper functions like:

setGasnet() {
  export GASNET_SPAWNFN=L
  export GASNET_ROUTE_OUTPUT=0
  export CHPL_GASNET_CFG_OPTIONS=--disable-ibv
  export CHPL_TARGET_ARCH=none
  export CHPL_COMM=gasnet
  export CHPL_QTHREAD_ENABLE_OVERSUBSCRIPTION=1
  export QT_AFFINITY=no
  export GASNET_QUIET=1
  export CHPL_TEST_TIMEOUT=500
}

and then I'll edit my ~/.bashrc to call setGasnet before I do a gasnet paratest.

This is admittedly awful as it's silly that I'm editing my ~/.bashrc and it also bakes in settings that nightly happened to be using at the time I made my helpers.

The long term solution here is probably to have jenkins jobs where you can just test any random config with your branch.

That's likely to be a lot of work, so in the short term we could come with a simple wrapper script to help test different configs.

My current thoughts are make a testChpl (or whatever) wrapper script that can take arguments for --config and --repo. There should be a list of valid configs (gasnet, gasnet-fast, gasnet-fifo, numa, no-local, valgrind, etc.) and those configs will map down to calling setup scripts in util/cron so that we do what nightly does. Oh, and actually we should probably have the config names match the ones that the regression mails use.

The script should take care of cloning an appropriate repo/branch (or reusing the current repo), doing a fresh chapel build (clobbering if need be), and kicking off a paratest.

[ronawho]

@mppf I think this captures the gist of what you brought up in the Monday meeting. Feel free to add any misc thoughts you may have.

[mppf]

For a script-based solution, I don't think it needs to do a fresh checkout or a clobber, but I was hoping it would at least make sure the tested configuration is built (i.e. with 'make' calls that possibly do nothing).

[riftEmber]

The short-term component of this is partly addressed by Andy's prequal tool, which provides a paratest-launching GUI that allows selecting between different configurations as well as creating custom configurations. However it doesn't cover all the example configurations listed in the OP or that exist in nightly. It also isn't re-using util/cron setup scripts AFAIK which means adding to or changing configurations requires manual work.