Run SciUnit tests at specified intervals

[gsarma]

SciUnit tests are fundamentally different than ordinary unit tests, and in particular, the underlying models are unlikely to change as frequently as other code. Also, if at some point in the future running the tests themselves becomes computationally expensive, this could slow down the build times unnecessarily.

One solution is to have SciUnit tests run at specified intervals, rather than with every build.

@rgerkin @cheelee @slarson

[rgerkin]

@gsarma @cheelee @slarson I now have the sciunit tests running successfully every time a push is made to the sciunit-travis branch, and creating notebooks summarizing the test with figures (which still need persistent storage as described in #185. I see two main ways forward: 1) Make the tests only run when the sciunit-travis branch is updated, for example when everything from other branches is pulled in. This makes the sciunit-travis branch a dedicated testing branch.
2) Merge the sciunit-travis branch back into the main branch, so that the tests run (and store notebooks) every time the main branch is updated.

With (1), we only run tests when we want (by explicit pulling into the sciunit-travis branch), which could be good or bad. One downside of (2) is that our persistent storage could get bloated since we will be storing test results even when the change to the repo is trivial, e.g. a typo fix to the documentation.

Which path should we take?

[gsarma]

Just wanted to circle back to this, particularly since this exact issue was raised by one of our reviewers on the testing paper.

My inclination is that option (1) may be the way to go because running these tests every time the main branch is updated is probably too computationally expensive. If I'm understanding it correctly, it also requires developers who are working on updating models to know to update the sciunit-travis branch, otherwise new models won't get tested.

We may also just need to have a project wide policy of periodically validating all of the models on main by uncommenting test code or something like that.

@gsarma @cheelee @slarson @VahidGh