search

flux-framework / flux-test-collective

Holistic system testing and CI for multiple flux-framework projects
GNU Lesser General Public License v3.0
0 stars 2 forks source link

cd: make our builds globally accessible on the machines we test #4

Open wihobbs opened 11 months ago

wihobbs commented 11 months ago

From a slack discussion with @garlick, @grondo, @trws, and Ryan:

Tom noted:

a bunch of the app teams are wanting to use features from a newer flux version in CI on lassen [...] They can build one, but there's common CI infrastructure used across about a dozen projects now and also I've heard from folks in 2-3 other directorates who really want pretty much the same, so having a single copy we can send people would be useful.

And Jim said:

I think Ryan installed the older version that's out there. I was working towards a package update but ran into difficulties with the packaged versions of pmix and hwloc being too old. We can get around that in a side installed version I guess. In theory we should be able to get flux-pmix working there and then launch spectrum properly. flux-pmix also includes a plugin for the broker's PMI client so we should be able to run the broker directly with jsrun. Or that's the theory. I've not been able to get very far testing this on sierra. So in summary, we need side installed hwloc, openpmix, flux-pmix, flux-core, and flux-sched. And we need to make sure none of that causes problems for spectrum which is also effectively side installed. it'd be nice to get a newer version built. 0.45 is not that old but it is 10 releases behind and a lot has been done. plus flux-pmix.

To which Tom replied:

The main thing would be a flux-core, flux-sched, and flux-pmix with a cuda-enabled hwloc if it were me, I'd probably set up a spack env with flux-core flux-sched flux-pmix@main ^hwloc+cuda

At which point I said:

Idea here: is there any way GitLab could help automate some of this heavy lifting? Our runners already build core every night (for a service user only to test) on tioga, corona, poodle, and quartz.

The first step would be to find a globally readable space to put the builds. We'd also need to get the general make check CI testing working with sched and pmix.

It also sounds like we could do 2 CD builds: one for each tag in core, and one that built nightly with the absolute latest features we've just landed.