feat: CI build and test pipeline

[wdconinc]

This PR adds a build and testing pipeline, using both gcc and clang as reference compilers. It runs the mock data generator and subsequent analysis. No validation on the analysis results is performed. This PR also adds a clang-tidy analysis for modified code segments (presumably some suggestions will appear on this PR).

TODO:

• [ ] merge #13 and rebase, as this PR is built on top of it.

[wdconinc]

Awaiting actions from being enabled here, https://github.com/wdconinc/japan-MOLLER/actions/runs/9166948693 has the intended output.

[wdconinc]

@paulmking @hansenjo FYI

[hansenjo]

JLab's cpu allowance for GitHub workflows is very limited. The plan is to run CI on the new site code.jlab.org, which is in beta. We should hold off with this.

If you have time, look into setting up web hooks for triggering CI runs at JLab. The SciComp folks are supposed to research this, but at the rate they are going, it might be a long while. The other halls and probably EIC are also interested in such a GitHub->JLab link.

[hansenjo]

Also, I suggest we keep our CI workflows independent of CERN software collections. As a rule of thumb, make runtime environments as standalone as possible or at least only dependent on things we control. (JLab does export application software via CVMFS, too.)

[wdconinc]

JLab does export application software via CVMFS, too.

Sure, where is the documentation for that?

[wdconinc]

allowance for GitHub workflows is very limited

Irrelevant for PRs where this is going against the submitter's CPU allocation.

[wdconinc]

The other halls and probably EIC are also interested in such a GitHub->JLab link.

We are already using this extensively for EIC (and I am very familiar with all the intricacies). It won't solve the problem that PRs are untrusted code that doesn't have access to the secrets to trigger pipelines on code.jlab.org.

There is also the fundamental problem that as long as JLab cannot give my grad students computer accounts, they (and I) won't do anything that locks access to collaboration software to needing to have a JLab account.

So, my suggestion is to merge this and if code.jlab.org goes out of beta we can evaluate it then.

[hansenjo]

JLab does export application software via CVMFS, too.

Sure, where is the documentation for that?

Not sure. Why not file a helpdesk ticket along with a suggestion to add the documentation to the Knowledge base.

[wdconinc]

@wmoore28 Can you point to the kb docs on how to use JLab application software over cvmfs? All there appears to be is https://jlab.servicenowservices.com/scicomp?id=kb_article_view&sys_kb_id=4aa518981b7dc110a888ea4ce54bcb79 which doesn't actually help in loading any software environment.

[hansenjo]

allowance for GitHub workflows is very limited

Irrelevant for PRs where this is going against the submitter's CPU allocation.

I wasn't aware of this (minutes being "paid for" by the submitter). At least at some point in the past, this was different. If this is really so, some of the GitHub related discussions at JLab no longer make sense.

Could we then find ourselves in a position where a CI job doesn't run because the particular submitted (presumably a pretty active person) is out of their minutes? What IS the allowance for free personal account anyway?

[wdconinc]

Could we then find ourselves in a position where a CI job doesn't run because the particular submitted (presumably a pretty active person) is out of their minutes?

You are never "out of minutes". You get 20 job slots at 4 cores. If they are all in use then new jobs are queued until some jobs finish. I am using GitHub actions extensively and I have never experienced the limits to be ... limiting.

Ref. https://docs.github.com/en/actions/learn-github-actions/usage-limits-billing-and-administration#usage-limits

[hansenjo]

The other halls and probably EIC are also interested in such a GitHub->JLab link.

We are already using this extensively for EIC (and I am very familiar with all the intricacies). It won't solve the problem that PRs are untrusted code that doesn't have access to the secrets to trigger pipelines on code.jlab.org.

There is also the fundamental problem that as long as JLab cannot give my grad students computer accounts, they (and I) won't do anything that locks access to collaboration software to needing to have a JLab account.

Well, the access to the code would hardly be "locked", would it?

My understanding was that CI jobs on code.jlab.org linked to GitHub projects would not require the submitter to have an account on the CI system. The SciComp folks aren't very forthcoming with details. Obviously, there are a few things we need to clarify and/or lobby for. It's good that you bring this up.

I'm not sure why your students can't get JLab accounts. If they work on JLab projects, they should be able to. Access to code.jlab.org should eventually be available through federated login anyway (it isn't yet).

So, my suggestion is to merge this and if code.jlab.org goes out of beta we can evaluate it then.

If minutes are not an issue, sure. Then again, I'm not the maintainer here. I'll let Paul decide. In any case, we might want to squash those various trial and error commits, maybe rebase the whole thing into a one or two commits.

[hansenjo]

Could we then find ourselves in a position where a CI job doesn't run because the particular submitted (presumably a pretty active person) is out of their minutes?

You are never "out of minutes". You get 20 job slots at 4 cores. If they are all in use then new jobs are queued until some jobs finish. I am using GitHub actions extensively and I have never experienced the limits to be ... limiting.

Ref. https://docs.github.com/en/actions/learn-github-actions/usage-limits-billing-and-administration#usage-limits

Thanks for the link. It seems like the policies have changed since I last looked at this. Interesting.

[wdconinc]

we might want to squash those various trial and error commits, maybe rebase the whole thing into a one or two commits

I was assuming squash merges would emerge as the default policy, so yeah this should be squash merged. If you have a CI validation setup, then regular merging isn't as useful since you cannot guarantee individual commits pass validation.

[wdconinc]

Rebased on current main.

[wdconinc]

I can add that having a dual strategy of both gitlab and github pipelines is the most useful. Even if code.jlab.org becomes a reality, we would still want to maintain the ability to run tests on github. This is just a matter of redundancy because otherwise you are dead in the water when either github actions or code.jlab.org goes down. For EIC we have found that to be very important for throughput (because things like code.jlab.org will go down when you need them most, since our needs are going to be correlated to the load on those systems).

[hansenjo]

I can add that having a dual strategy of both gitlab and github pipelines is the most useful. Even if code.jlab.org becomes a reality, we would still want to maintain the ability to run tests on github. This is just a matter of redundancy because otherwise you are dead in the water when either github actions or code.jlab.org goes down. For EIC we have found that to be very important for throughput (because things like code.jlab.org will go down when you need them most, since our needs are going to be correlated to the load on those systems).

That sounds good. Regarding code.jlab.org, let's wait and see (and I will lobby for a community-friendly configuration).