Add Automated Code Testing

[ewanwm]

Would be very nice to have automated code testing which can be integrated into existing continuous integration pipeline. I think the best place to start with this would be some very basic unit tests for e.g. the spline code. For us I think what this would look like would be something along the lines of:

• Generate some very basic sets of dummy spline files (like a file with all flat splines, one with all with empty (single knot) splines, one with all linear 2 knot splines... etc.)
• Create unit test executables where we pass each one through each of the spline types and check that they all give the expected behavior (check all the knot values are what they should be, like check all the monotonic splines are actually monotonic, etc.)

Once these are added for the basic components (which might take a while) we could also think about some higher level integration tests. For us I think that might look like actually running some LLH scans, sigma vars etc. and eventually MCMC using the super basic dummy splines and some vv simple samples and checking that the outcome is as expected.

There are a bunch of tools for doing these tests automatically. One of the simplest is maybe CTest which is super easy to use and even comes packaged in CMake.

This kind of testing has a bunch of advantages, mainly:

• If included as part of continuous integration it means that you can very quickly catch new bugs that some new feature might cause before it makes it into develop
• It can help finding old bugs by testing weird edge cases that wouldn't come up in an obvious way when you're actually running the code.
• It can point you exactly to where problems are based on which tests fail, assuming tests are granular enough
• It can help give peace of mind when people are installing code as they can just run the tests and make sure everything is working as expected

If we're feeling extra fancy there are even tools for tracking the test coverage of the code such as codecov so that we can check how much and how well the code is tested.

I'm mainly just throwing this issue out there to get peoples thoughts on this and see if anyone has thought about this already/ has already started looking into it

[KSkwarczynski]

Fully agree,

For covariance it is also quite easy to run CI within core.

If we want Fitting Algorithms we can test them using only covariance objects.

SamplePDF have to many experiemnt sepcyfic bits.

[ewanwm]

For samplePDF i was wondering if it makes sense to have some very minimal "dummy experiment" implementation thats used just for testing. Maybe in some kind of testing specific library so it doesn't get included in MaCh3::All. But you definitely know the code much better than me so I will defer to you on whether or not that is sensible 😅

[KSkwarczynski]

Indeed there are such plans :) https://github.com/mach3-software/MaCh3/issues/44

Plan is to implement "Tutorial". Which would be sort of dummy samplePDF using Nuisance Tree (or other formats). This would both be hepfull for learning but also CI.

You can imagine as new "repo" which CPM MaCh3 and proceed with tests.

These are only plans right now :(