Manage configuration as JSON

[avanier]

This PR suggests a few changes :

• It changes managing the configuration from hcl to json which allows users to arbitrarily change all parameters of server configuration without requiring new PRs on this repo for every setting we don't intend to manage.
  • This change is very breaking as it moves around about half the configuration namespace.
• It changes the vault user from root to a configurable user, vault by default, and re-owns all the appropriate files
• It prevents the self-signed certificate generation script from silently failing under several conditions
  • For example, setting vault.self_signed_cert.country=Canada instead of vault.self_signed_cert.country=CA would silently fail. Yes, sometimes we do derpy things.
• It prevents self-signed certificates from being regenerated if they've already been generated

This PR has a lot of overlap with #7 and would fix #8 .

If this gets merged, please squash.

[myoung34]

If you can get the tests to pass I wont be against merging it after s ome manual validation

[avanier]

Ok, the whole SELinux thing was a lot of fun. Also, this is how you become good at SELinux... with self-inflicted fun.

[avanier]

Ok, this is not SELinux. There's just something different about journald in Docker which logs differently than in a VM. I'm not sure what to do here. I just can't get that test to pass in Docker.

This means we can discard 8965b8a.

[myoung34]

It fails on centos7/aws linux, so my initial guess is that one of the mods in the PR is failing it, I'm just not sure which one

[avanier]

My current speculation is this has to do with the fact that in Docker, systemd is owned by PID 1. Somehow that would prevent a user that is not root from forwarding its logs through systemd to journald. If I set the user to root, the test passes. But this is only in Docker. I have an actual deployment of this on CentOS 7 in AWS working as expected. If I test this with VirtualBox or vagrant-libvirt (KVM/QEMU), the test passes.

The behaviour experienced in Docker, is that when the process starts, it logs to the journal under the systemd PID (1), and the actual application standard error and standard out will be forwarded to the journal under its own PID (as it should), but that will get logged in a different unit, which causes the test to fail as the exepectation is to find the logs under vault.service, which references to its starting pid of one, which shows up the logs for the process being started by systemd. The lined that's regexed for is to be emitted by standard error.

TL;DR, application logs, but not in the right place in Docker, which fails the test.

I offer the two following options :

• I refactor .kitchen.yml to run that suite under Vagrant VirtualBox, which will increase the Travis build time significantly, but will provide results representative of actual deployments.

• I remove the test, which significantly reduces the amount of tests.

  ¯\_(ツ)_/¯

[avanier]

Oh, and the whole SELinux thing is garbage.

[avanier]

There's also option three, where everyone runs Vault as root, but that would foil all of my hardening efforts with SELinux and auditd. 😞 (That's where my SELinux garbage came from.)

[myoung34]

I'm definitely not a fan of removing tests. The way I see it: tests currently pass on master. Adding code and removing tests because they wont pass is a major smell

[blbradley]

Vagrant is not supported on Travis CI.

[avanier]

Another possibility could be to change the test. Curling /sys/health on a dev server would show the vault as unsealed and initialized. It's less robust of a test, but would serve the same purpose.

The other alternative would be to grep the entire journal instead of just the unit.

[avanier]

Stale, closing.