research-software-reactor / code-testing-for-researchers

Supporting in-cloud testing for researchers
Apache License 2.0
2 stars 0 forks source link

We need burst capabilities for large performance tests #2

Closed tmbgreaves closed 5 years ago

tmbgreaves commented 5 years ago

Research software frequently needs to make sure that changes to the code don't adversely affect real-world examples of code use in terms of accuracy or performance. Often a large example problem can be reduced somewhat, but doesn't fit on standard CI worker hardware. Tying in to HPC instances can provide capability, but not guaranteed stable performance characteristics, and purchasing dedicated large hardware for CI is prohibitively expensive. Using cloud resources to occasionally burst a large instance may be a good solution, but can be expensive if not well constrained.

Produce learning materials to introduce the use of on-demand cloud resources in CI and how to monitor their use and ensure costs don't run away.

tmbgreaves commented 5 years ago

This is again well provided for by https://docs.microsoft.com/en-us/azure/jenkins/jenkins-azure-vm-agents