Better BOSH Targeting

[jhunt]

As an operator, I would like to spend less time configuring Genesis CI/CD pipelines, and minimize the amount of duplicated data in said configuration.

To achieve this, we believe the following should suffice:

1. Genesis should start tracking metadata about each deployment it has dealt with in the Vault, under secret/genesis. For example, BOSH director deployments will want to provide their path into the Vault (to handle params.vault customizations), as well as the URL of the director:

$ safe get secret/genesis/some-env-bosh
---
url: https://10.0.0.4:25555
base: secret/some/env/bosh

This will allow other deployments to reference this director, and its secrets, as needed. In effect, this is an extension of the core BOSH "links" concept, using Vault as a configuration server, but operating across BOSH deployments.

genesis deploy will be responsible for updating the entry under secret/genesis after every successful deployment, whether by interactive operator use, or by concourse pipeline.

Each kit will be responsible for populating other keys, besides base (which Genesis will get from params.vault, defaulting to the normal default). These values will be placed in the new manifest top-level genesis key:

genesis:
  url: (( concat "https://" params.static_ip ":25555" ))

This will be imported into Vault via safe import, so the keys under genesis should only be a single level deep (no nesting!). If the kit supplies a genesis.base key, it will be overwritten by genesis.

2. Environment deployments will continue to specify their BOSH director via params.bosh, which will be interpreted as a tertiary path component in secret/genesis/$env-bosh. For example:

params:
  bosh: us-east-1-prod

would be interpreted as secret/genesis/us-east-1-prod-bosh for lookups.

3. Add a new command called genesis login that will do the required bosh CLI calls to set up an alias, using the correct parameters:

bosh -e $URL logout
bosh -e $URL --ca-cert "$CA" alias-env $ENV
bosh -e $ENV env
bosh -e $ENV login

This will make life easier on operators, who are tired of dealing with BOSH's quirks on the command-line.

4. For CI/CD use cases, the ci.yml will pick up a new top-level param called bosh_client (or bosh-client, whatever is consistenter) that will identify the name of the UAA client to use when talking to BOSH. It will default to concourse.

For each BOSH environment that concourse has to deal with (via the env's params.bosh) it will look up the client secret in Vault by finding the base under secret/genesis/$env-bosh, and then look up $base/users/$user:password

This should remove a fair amount of logic from repipe and move it into the CI commands themselves (ci-pipeline-deploy)

[dennisjbell]

The other benefit of this (for the benefit of others reading this) is that it gets away from having users have to maintain their own collection of .bosh/config aliases and having different names all over the place.

[rkoster]

Would this also mean genesis new some-env would lookup bosh creds under secret/genesis/some-env-bosh or would I need to do a genesis login first?

[dennisjbell]

The genesis login would be for users to be able to create and use a local alias from the bosh creds stored in vault. Genesis itself would use these creds independently for its own purposes, not requiring a bosh env alias.

As an aside, the genesis login functionality is already a reality using the latest version of genesis and the bosh-genesis-kit, using the genesis do <env> login incantation.

[mrferris]

Delivered in v2.8.0