switch `vault` in nci to `vault-nci`

[pivotal-bin-ju]

We've achieved a fantastic milestone - moved our ci from bosh to helm. And vault is enabled to the nci environment as well.

The vault has auto-unseal enabled, that is cool. But with the file system backend, we are lacking of backup. That is why we created a new instance vault-nci, which has cloudsql/postgres as the backend, and the auto-backup is enabled.

Here is the plan for the switch:

• [x] Get the PR merged
• [x] terraform plan to see the grap. If the gap exists, terraform apply.

• [x] Do the data sync check between the vault and vault-nci. We could leverage the migration tool to sync the data. You could either offline mode (export first, then import) or the online mode. Here is the example for online mode.

  • port-forwarding the both vault if they are not reachable from your host.

  • run the verify command. e.g.:

    go run main.go -source-addr="https://localhost:8201" -source-token="7b35793c-a809-7406-b14c-73611506626a" --target-addr="https://localhost:8200" --target-token="s.CAAuHWSvkHkdmmCLS123cT03" --dry-run=true --debug=true --root-path=/concourse/main --root-path=/concourse/shared

    if dry-run=true, the tool does not apply the change to the target vault, just show you the differences.

• [x] secret update. The make commands rely on the secrets from lastpass. For our case it is hush-house-values-vault. To make the vault-nci worker, we have hush-house-values-vault-nci. So what we should do now is:
  • rename hush-house-values-vault to hush-house-values-vault-obsolete
  • rename hush-house-values-vault-nci to hush-house-values-vault
• [x] delete the pod vault-nci/vault-nci-0
• [x] delete the pod vault/vault-0
• [x] helm deploy vault.
  • make creds-vault
  • make deploy-vault
• [x] verify the new vault. Please see the readme page.

[pivotal-bin-ju]

WDYT @cirocosta ?

[cirocosta]

Hey @pivotal-bin-ju,

delete the pod vault/vault-0

Was the idea of the deletion to just restart the pod?

If not, I think this is not really what we want because by doing that removal, we'll only get rid of the current instantiation of the pod, having another coming back right after:

NAMESPACE  NAME              
vault      StatefulSet/vault 
vault      ├─ControllerRevision/vault-bb9698cb7
vault      └─Pod/vault-0     

if the goal is to get the configuration change, the next steps (helm upgrade wht a new config), then that should cover the deletion of the pod, etc along with the transition to whatever new configuration we want :grin:

[pivotal-bin-ju]

@cirocosta , while the pod exists, we run helm command to redeploy the pod, it complains the instance is already exists. Anyway, I will try make deploy, if it doesn't work, then I will helm delete the pod.