Automate the deployment of the portal on Azure

[jonc125]

Ideally merges to development should update the staging host, and to master the production host.

See also https://github.com/UCL-CloudLabs/deployer/issues/11 about the private key(s) needed by staging & production systems. More generally we need to figure out how secrets can be stored and automatically decrypted on the portal systems.

[jonc125]

• [ ] At present the relevant repositories are checked out in our home folder. We'll want to put this in a system location (e.g. /opt) and create a GH token allowing read access.
• [ ] Look into using ansible for specifying the machine setup. Or possibly a declarative alternative such as SaltStack in masterless mode or over SSH to make upgrades/changes easier.
  • There's an argument that we ought to use puppet for consistency with other services, but that possibly requires a separate puppet master that the Azure VMs can connect to... Although as with SaltStack it's in theory possible to do master-less puppet. Just a question of how you manage secrets!
  • There are salt formulas for postgres, apache, RabbitMQ to simplify setting these up, but not (yet) for Celery, Terraform
• [ ] Initially we can run ansible manually rather than automatically on merges
• [ ] ansible-vault may be sufficient for our purposes to handle secrets

[ageorgou]

An update on this:

@raquel-ucl and I tried the Azure Web Apps framework, which is meant to offer suitable development environments and linking with version control systems (either local repositories or GitHub). You can create a Web App resource (through the portal), but it only offers NodeJS, Ruby and PHP frameworks. There is a Flask resource which should work, but it only offers Python 3.4 (or 2.7), and has a problem with the package installation (specifically during Jinja installation due to an async keyword, introduced in Python 3.5). There is apparently a way to use more recent versions, but I couldn't get it to work. The latest attempts are in the docker-sample repo for future reference.

There is also the App Service, which is supposed to offer similar functionality and support Flask more directly, but we couldn't figure out whether we need a new account/subscription, nor make one, so we couldn't explore this more.