search

Azure / azurehpc

This repository provides easy automation scripts for building a HPC environment in Azure. It also includes examples to build e2e environment and run some of the key HPC benchmarks and applications.
MIT License
123 stars 65 forks source link

Autoscaling configuration #174

Closed jedbrown closed 4 years ago

jedbrown commented 4 years ago

Has someone worked out how to configure an autoscaling cluster that can use up to N compute instances (spun up when submitting with qsub and automatically spun down upon completion)? It looks like all the examples are demonstrating one-off bigger jobs rather than an on-demand service for a small group with testing/debugging needs (which I think has a substantial audience among researchers and open source developers).

It'd also be really useful for upstream Azure to document/monitor the distribution of time to provision resources. Some colleagues in the past observed it to be significantly higher on Azure than a competitor, but it takes human resources to make representative tests and summaries like this are incomplete and may not be representative of the instance types that an HPC user cares about.

xpillons commented 4 years ago

Thank you for your feedback. As of today, autoscaling is not implemented in azhpc, it does thru CycleCloud which can also be deployed thru azhpc. There will be certainly enhancements done in the futur.