Add cloud parallelisation capability

[martintoreilly]

Description

Create a script to deploy a set of simulations to Azure. Other cloud providers are not in scope for this issue.

Notes

• Should provide a single parent script that spawns child processes on either the local machine or Azure.
• Look at Azure Batch
• Use the Microsoft Data Science VM
• Should we use a Jupyter Notebook or a plain Julia script? @tlienart What are your thoughts on this?
• How to handle authentication while ensuring that the risk of credentials being committed to Github is very low?
  • Look at using the Python Azure Active Directory Authentication Library, which can handle 2FA.
  • Alternatively look at the Python Keychain library, which supports OSX, Windows and some Linux local credential stores.
  • In either case, can use PyCall to call Python from Julia.

[tlienart]

Re jupyter notebook vs julia script, I vote for julia script and keep notebooks for analysis etc.

the rest is looking good 👍

[martintoreilly]

Authentication to Azure

Borrow functionality from the Azure CLI 2.0 client. Note that the the Azure CLI 2.0 client is a Python app. To access the azure-cli modules, install them using pip install azure-cli. This will also install the CLI client itself, which can be run using the az command.

We will also need to install the matching v2.0 azure libraries. To do this, run pip install --pre azure (the --pre is required as v2.0 is still in pre-release). If you have already installed the non-preview v1.0 azure libraries, you will need to uninstall these using pip uninstall azure (you can check the version you have installed by running pip freeze.

See below for a Python snippet that interactively authenticates to Azure accounts, including ones with two-factor authentication enabled.

from azure.cli.core.prompting import prompt_pass, NoTTYException
import azure.cli.core.azlogging as azlogging
from azure.cli.core._profile import Profile
from azure.cli.core.util import CLIError
import sys
import logging
logger = azlogging.get_az_logger()

ch = logging.StreamHandler(sys.stdout)
ch.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
ch.setFormatter(formatter)
logger.addHandler(ch)

profile = Profile()

res = profile.find_subscriptions_on_login(True, None, None, None, None)