Figure out deploy strategy for CouchApp

[gordonbrander]

This PR made it possible to host the UI as a couchapp on the Raspberry Pi https://github.com/OpenAgInitiative/openag_ui/pull/127. You can push the built UI to any CouchDB install, and it will be hosted and delivered by couchdb. The only thing left is to figure out a deploy strategy for automatic install.

Here's an initial idea... Add the following to the install script:

1. Publish built dist. (Check in /dist directory, or record location of release zip files in a manifest`)
2. Download dist files to tmp dir
3. Install with script (couchapp, ideally)
4. Trash tmp dir
5. Write to hosts file

This still leaves the question of what to do for Docker. Make it part of the Docker install script? Maybe we keep the dist dir around and push every time?

[gordonbrander]

Alternatively, could host the UI in a publicly available CouchDB install, then replicate the whole thing.

[gordonbrander]

Unlike the Node tool, the Python tool for couchapps is pretty specific about the directory structure required. This is unfortunate. It also has a lot of project scaffolding features we don't need.

We don't need to do much except put a few files as attachments. I'm going to look into whipping up a quick Python script.

[gordonbrander]

@LeonChambers I've created a tool to push a folder full of files to a design document. The idea is to make installing the UI to Couch part of the setup process.

During install:

1. Clone compiled frontend files to tmp dir
2. pip install this tool
3. load files
4. trash tmp dir.
5. modify hosts file

Do you have any strong feelings about that?

[LeonChambers]

I wrote something similar here as part of openag_python to load the design documents from the repo. It doesn't handle attachments though. It would be nice to merge these two things.

But I'm definitely fine with this strategy in general.

[gordonbrander]

@LeonChambers Yeah I saw that. I thought it might be useful to extract this into a general utility that can handle attachments + views, lists, etc.

[LeonChambers]

Have you looked into couchapp? It also seems to do exactly what we want. I'm not sure we need our own utility for this.

[gordonbrander]

@LeonChambers extensively. It requires an opinionated directory structure that I'm not wild about -- would require restructuring the UI build stuff. We could go that way, but since we already have similar requirements for openag_python, I thought it would be good to have them go through the same codepath.

I don't have a very strong opinion about this, but the code to push to CouchDB is trivial. couchapp adds a lot of bells and whistles we don't use like project scaffolding.

[LeonChambers]

Ok, that makes sense. Then I guess the next step here is to modify openag_python to use this new tool so that we don't have duplicate code.

[ghost]

@gordonbrander and @LeonChambers, have you guys considered approaching this as part of using a configuration management tool to build a project-wide deployment system? SaltStack seems like a perfect fit.

Compared to building more custom scripts, using SaltStack would let you manage things at a higher level of abstraction with more flexibility, repeatability, and readability. Most--perhaps all--of the building blocks you'd need to make the PFC2 setup process automatic and versionable are already implemented. Also, a big benefit is that better automation reduces the need for extensive documentation.

---

Here's how I'd go about writing salt states to build and deploy the CouchApp on a development workstation:

1. As part of setting up the overall deployment system, write a bootstrap bash script to install and configure salt as a masterless minion.
2. Use the states.pkg.installed module to ensure that the nodejs, npm, and nodejs-legacy packages are installed.
3. Use the states.npm.installed module to install grunt.
4. Use the states.git.latest module to clone OpenAgInitiative/openag_ui into a non-temporary directory, or, if the directory already exists, to do a git fetch instead. States are just YAML templates, so it's easy to use variable substitution to specify the branch or revision git.latest should use. Leave the cloned directory in place so the state will run faster the next time around.
5. Use a combination of states.npm and states.cmd.run to install and build openag_ui into static files.
6. Assuming CouchDB has already been configured using some other state, use states.cmd.run to deploy the static files with grunt couchapp_deploy.

---

If you want to see what salt states look like for a simple deployment system, here's some stuff I made last year for setting up a minimal web app with nginx on a Debian 8 gcloud instance:

• Bootstrap scripts to install and invoke salt: https://github.com/wsnook/salt-webapp-setup
• Salt root filesystem with states and configuration: https://github.com/wsnook/salt-webapp-roots

[gordonbrander]

@wsnook this looks very cool, but it also looks tailored to the problem of cloud servers. Do you have insight into how it would look to use for setting up a Raspberry Pi?

[ghost]

...looks tailored to the problem of cloud servers

Right... I'm guessing you're referring to how my example used separate master and minion servers. I should have expanded on the "masterless minion" part that I just mentioned in passing (see masterless quickstart docs).

First, salt jargon is weird, so here's a quick glossary:

• minion: in this context, a PFC2's Raspberry Pi
• state: a declarative specification of one facet of desired configuration for a minion
• apply: invoke whatever magic and dependencies are needed to achieve a desired state
• master: in a cloud or data center context, the normal place a sysadmin would go to boss minions around

Basically, masterless salt on a Raspberry Pi would look like:

1. Forget about the master. We don't need one.
2. Put the state files in /srv/... on the Pi minion instead of the non-existent master.
3. Invoke salt with sudo salt-call --local state.apply foo on the Pi minion instead of doing the cloud style sudo salt <target_minion_id> state.apply foo from a master.

SaltStack's architecture is based on a message bus. On any host managed with salt, there's a minion process that can manage the host by "applying" states--conceptually it's more declarative than procedural. The minions can join a master's bus and get information about which states to apply when from that master, or the minions can have their own state files. Normally messages to the minion process arrive over the network, but you can also send them locally with salt-call.

[ghost]

Oh, and for how masterless salt could be used to manage the whole OpenAg stack, I'm envisioning something approximately like:

1. Flash a clean Raspbian Jessie Lite SD card.
2. Boot a Pi with the new SD card and log in to a shell.
3. Copy and paste a bootstrap script from one of the OpenAg github repos that will minimally configure the Pi, install git and saltstack, check out the state files, and then invoke salt to apply the state for, "Make this Pi into a PFC2".