Regression Testing Management routines
regtest.py is the main regression testing script for AMReX-based applications. Typing './regtest.py -h' will give a verbose description of usage and setup. The regtest set of tools is fully documented in the AMReX Users Guide, but here's a quick overview of usage.
Quickstart:
Each regression test suite is defined in a separate configuration file. AMReX is distributed with an example such file in Tools/RegressionTesting
With example-tests.ini as an example, which builds tests for the Castro code.
a. Castro depends on the following addition git repositories: AMReX, Microphysics, wdmerger
b. In the config blocks of the .ini file, the locations of clones of these files to use for testing are set as "dir". There are blocks in the ini file for each; AMReX has its own special block, and Castro is listed as the [source] block, but any additional dependent repos are listed as [extra-XXX].
For each repo defined in these blocks, the tester script pulls the desired branch/SHA prior to building/running tests. It is probably best to have these as "extra" clones in a special area used solely for this purpose so that the tester can work on its own without confusing you.
Thus...edit the paths in the config file to point to local clones of each of these repos.
c. Parameters that define each regression test appear in labeled blocks below the repository blocks. The number of options allowed for specifying these tests will certainly grow over time; the -h option to the tester should list everything currently available.
d. Each labeled test will be built, run and compared to "benchmark" results. The results of these tasks will be formatted into a web page in a folder defined as "webTopDir" in the .ini file. Make sure that variable points to something you like and can access.
e. You need benchmarks to exist before the tests can work. Generate them manually with
./regtest.py --make_benchmarks "
f. Afterward, tests are run by typing
./regtest.py example-tests.ini