Some research software will require more testing capability than is available from a fully hosted and managed platform such as Azure Pipelines. In these cases, it's useful to be able to deploy a custom testing platform into the cloud. Whilst there is already a lot of detailed documentation about deploying custom CI to the cloud, it often makes assumptions that are very valid in the case of applications managed by people with a technical systems background but which are not well suited to research software projects.
Key considerations from a research software perspective are:
All required resources provided by cloud services
Simple deployment -- no systems management skills required
No ongoing systems management required for maintenance (host upgrades set to automatic, testing platform upgrades handled automatically or from inside the testing interface)
No state saved on the master host (it should be able to be discarded and recreated trivially)
All testing workers deployed on-demand
Testing platform internal state saved to cloud storage
Deployment/redeployment templated and possible from command line
Some research software will require more testing capability than is available from a fully hosted and managed platform such as Azure Pipelines. In these cases, it's useful to be able to deploy a custom testing platform into the cloud. Whilst there is already a lot of detailed documentation about deploying custom CI to the cloud, it often makes assumptions that are very valid in the case of applications managed by people with a technical systems background but which are not well suited to research software projects.
Key considerations from a research software perspective are: