Use HPC cluster management tool

[JiaweiZhuang]

CfnCluster might complement #4 , especially for a large number of ensemble runs. You will get HPC-cluster-like environment, with a job scheduler handling multiple compute nodes. But it takes much longer to launch (~10 min) than basic EC instances (~seconds), and is quite difficult for a new user to learn. Will try this after #4 is largely finished.

CfnCluster is probably an overkill for GC-classic with OpenMP-only parallelization. But we will eventually use it for GCHP.

[JiaweiZhuang]

Not sure if we can use the same AMI for both CfnCluster and basic EC2 instances. I'd like to avoid maintaining two separate systems.

[JiaweiZhuang]

Might consider AlcesFlight instead (together with Singularity #11) http://cloud-gc.readthedocs.io/en/latest/chapter03_advanced-tutorial/hpc-overview.html

[JiaweiZhuang]

AWS ParallelCluster seems the best choice right now. See https://jiaweizhuang.github.io/blog/aws-hpc-guide/