flux-framework / flux-test-collective

Holistic system testing and CI for multiple flux-framework projects
GNU Lesser General Public License v3.0
0 stars 2 forks source link

testsuite: gitlab ci cluster specific tests #6

Open wihobbs opened 1 year ago

wihobbs commented 1 year ago

As @grondo and @garlick pointed out in flux-framework/flux-core#5482, there are some tests we'd like to run in our CI on real clusters in addition to the normal testsuite. Some ideas (opening this issue with the hope others will add to this list):

chu11 commented 1 year ago

although lower priority than the above, would we want to test with different python versions that are not the standard installed python? To make sure all the paths and what not are setup correctly? i.e. user uses python 3.X from /usr/tce and we make sure that works with the python 3.X from https://github.com/flux-framework/flux-python?? I think all Python we test in CI is the standard install python. (Edit: and perhaps this also double checks that all python dependencies are installed for each version? )

Note, I'm not sure the flux-python from above is installed yet ... so that maybe be another thing.

wihobbs commented 1 year ago

Snippet of @grondo and my weekly mtg:

  1. We're going to set up a new repository for flux-framework testing only. Iterating on core for integration tests might get cumbersome, and we want to test other framework projects.
  2. We'll run a nightly test of flux-core (do what was done in flux-framework/flux-core#5429) and some other stuff, too, that will be discussed in the Issues section of that new repo.
  3. There will be a Mattermost channel where the status of the nightly pipelines is reported.