Update scripts for running the integration tests to allow for more control by
specifying which suites and types to run. This allows for much easier creation
of GitLab CI jobs that run specific tests by simply changing the options.
Update GitLab CI jobs to make use of the more fine grained control of the
testing scripts.
Update .gitlab-ci.yml
Switch to using service user account
Add two new stages:
New install-deps stage
Switch to using Spack Environments
Check if existing, otherwise create and install
Make a prereq to the unifyfs build/install job(s)
New clean stage and job
create a full_clean job that only runs on a schedule and cleans up all old
builds from the gitlab runner directory
Clean up default before_script to simply activate spack environment and print
status
Add after_script section to integration tests to remove repo/logs after
successfull jobs to save space. Print location of logs for failed jobs for
debugging.
Add quartz system-specfic jobs
Updates the system-specific jobs:
Create templates for making scheduled jobs
Move shared variables for each compiler-specifc jobs into templates
Add jobs to install dependencies that the build jobs depend on
Split integration tests for each compiler into multiple smaller jobs running
specific tests (writeread-posix, writeread-mpiio, read-posix, read-mpiio,
producer-consumer-all, and stage)
Add scheduled jobs for running stage tests across larger set of nodes
Additional updates:
Update testing docs
Add .spack-env/ directory and spack.yaml files for each system/compiler for
creating/installing/activating the spack environments and allow for easier
maintainence and replication
Motivation and Context
To allow for easier control over which integration tests to run and to avoid space/time issues
from long running integration tests.
How Has This Been Tested?
Run on power9 and x86_64 systems through GitLab CI
Types of changes
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Performance enhancement (non-breaking change which improves efficiency)
[ ] Code cleanup (non-breaking change which makes code smaller or more readable)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[x] Testing (addition of new tests or update to current tests)
[x] Documentation (a change to man pages or other documentation)
Checklist:
[x] My code follows the UnifyFS code style requirements.
Description
Update scripts for running the integration tests to allow for more control by specifying which suites and types to run. This allows for much easier creation of GitLab CI jobs that run specific tests by simply changing the options.
Update GitLab CI jobs to make use of the more fine grained control of the testing scripts.
Update .gitlab-ci.yml
Updates the system-specific jobs:
Additional updates:
Motivation and Context
To allow for easier control over which integration tests to run and to avoid space/time issues from long running integration tests.
How Has This Been Tested?
Run on power9 and x86_64 systems through GitLab CI
Types of changes
Checklist: