Change Vault backend to something other than filesystem

[deniseyu]

We used to use an old version of Vault in BOSH prod, which may have used the filesystem backend as a default option. While it technically works, we may want to shift to a less error-prone storage backend because it is all of our production secrets after all, and it would be quite annoying to have to recover from someone accidentally running rm rf /vault for example while SSH'd into the VM/container.

All of the possible backends are documented here: https://www.vaultproject.io/docs/configuration/storage/index.html

We don't really need highly consistent/replicated/sharded/etc persistence so a lot of these strategies are overkill, but I'd say that the one feature we could benefit from is being able to easily make snapshots for backing up and restoring.

We already use GCS so my instinct would be to just pick that one, but this story includes room for doing some investigation.

An annoying thing we will have to do once is perform a data migration into the new schema. As far as I can tell there is no officially-supported way to migrate data between different backends.

• [ ] Research and choose a new backend
• [ ] Backup current prod Vault data
• [ ] Set up the new backend
• [ ] Update operating docs github.com/pivotal/concourse-ops wiki
• [ ] Restore old secrets into new backend
• [ ] Decommission old backend

[pivotal-bin-ju]

Hi @deniseyu , Please see my comment on #44 here, would cloudsql be better in term of backup/restore?