Feature-Request: INI files for CLI arguments

[evanfarrar]

On the the bosh-bootloader team, we've had some issues with string interpolation in our pipeline of Terraform -> Bash -> YAML/Bosh manifest. Our tool essentially maintains a script like this:

#!/bin/sh
bosh create-env \
  ${BBL_STATE_DIR}/bosh-deployment/bosh.yml \
  --state  ${BBL_STATE_DIR}/vars/bosh-state.json \
  --vars-store  ${BBL_STATE_DIR}/vars/director-vars-store.yml \
  --vars-file  ${BBL_STATE_DIR}/vars/director-vars-file.yml \
  -o  ${BBL_STATE_DIR}/bosh-deployment/gcp/cpi.yml \
  -o  ${BBL_STATE_DIR}/bosh-deployment/jumpbox-user.yml \
  -o  ${BBL_STATE_DIR}/bosh-deployment/uaa.yml \
  -o  ${BBL_STATE_DIR}/bosh-deployment/credhub.yml \
  -o  ${BBL_STATE_DIR}/bbl-ops-files/gcp/bosh-director-ephemeral-ip-ops.yml \
  --var-file  gcp_credentials_json="${BBL_GCP_SERVICE_ACCOUNT_KEY_PATH}" \
  -v  project_id="${BBL_GCP_PROJECT_ID}" \
  -v zone="${BBL_GCP_ZONE}" 

This script stores the "state" of the director in a sense. What ops files it was made with, in particular.

The issues have motivated us to explore replacing that script with a YAML file, or something. This would make the invocation of create-env so trivial that we wouldn't bother documenting it as heavily anymore:

bosh --ini-config bosh-arg-config.yml create-env

The go-flags library already supports this and calls it "INI files":

[create-env]

; comments

o=ops.yml

v=variable=value

[delete-env]

o=ops.yml

But that could just as easily be YAML, because this is Bosh after all:

---

create-env:

  o: "ops.yml"

  v: variable="value"

delete-env:

  o: "ops.yml"

Given this file above, you could also supply it to delete-env and have an effect:

bosh --ini-config bosh-arg-config.yml delete-env

@cdutra is already comfortable guiding our team on implementing this, and we could PR it, and @freddesbiens from the Bosh team has shown at least some interest in this direction so maybe we could get it merged as outline above, too.

However, we're interested in feedback to be better:

1. Help us come up with a better name for this.
2. Help us think of other usecases, like documenting a full deployment example more succinctly in cf-deployment. @heyjcollins
3. Help us think of how this format could be extended with more metadata, like maybe this could assist users in determining what ops files work with what other opsfiles, or maybe you could use these to prompt the user for input and make decisions kind of like outline in the Bosh bundles proposal

[evanfarrar]

One general benefit to this format is that it can be parsed more simply than Bash can be parsed. Historically, we used to accept a single ops file for customizing the director in BBL. When we looked to iterate on that feature to allow multiple ops files, we would have had to re-implement the flag parsing logic of Bosh itself to insert the new ops files arguments in the correct location in the invocation due to ops-file ordering issues. Here, flags are supplied in a completely machine-readable format.

[aegershman]

Forgive me, might be straightforward, but can env parameters still be interpolated using ini syntax? e.g.:

# bash snippet
bosh create-env ... ... \
  -o  ${BBL_STATE_DIR}/bosh-deployment/gcp/cpi.yml \ ...

# ini semi-equivalent
[create-env]


o=${BBL_STATE_DIR}/bosh-deployment/gcp/cpi.yml # <-- will BBL_STATE_DIR still be interpolated?

[cdutra]

@aegershman That's a good question. I haven't tested with bosh provided environment variables. With bbl environment variables it should be able to interpolate them, however our internal code would have to handle this because it would directly shell out to the bosh cli. About specific users variables, I don't believe so. Additional code would need to be written for that.

Thank you for bringing up this point.