Ability to use configuration secrets from in-memory instead of disk

[beneshed]

Feature Description

Currently encryption keys and TLS private keys are written to disk.

My proposal is to offer configuration options to pull secrets from memory or use more standard secure formats to store secrets.

Use Case(s)

I'd really like to not have a secret, plaintext written to disk.

There are two features where critical information is written to disk

1. Encryption key for Consul gossip protocol
   • https://www.consul.io/docs/agent/options#_encrypt
2. TLS Client/Server Auth
   • https://www.consul.io/docs/agent/options#key_file (this is real crucial part)
   • https://www.consul.io/docs/agent/options#cert_file

Gossip Encryption

Ok first I was frustrated to see that the key couldn't even be introduced through a environment variable or cli flag, but then I saw the keyring command

I thought, "Wow awesome ok awesome, on initial deploy i'll use one gossip key and then rotate so that the plaintext key won't be there", but then I dug into the code and saw that it uses a "keyring" file that is literally just a plain text json encoded file with keys stored on disk

So that defeats the purpose of trying to keep it off disk

I propose the following:

1. Being able to do the initial encryption key via some type of in memory function call
   • This could be done via Hashicorp Vault, AWS KMS/Secrets Manager, Azure KeyVault, GCP Key Manager/Secrets Manager
2. I'm cool with storing the key on the disk as a cache or backup, but make it encrypted. Standards wise there's PKCS12, but that's really made for certs not just secret strings
   • Why not use standard encryption like AES-256-CBC or something like NACL/secretbox which go supports both natively

Private Key of Client/Server TLS Auth

The private key file is plain text on disk. Cool you can use file permissions and hope for the best. It would be really nice to support PKCS12 files to load up the private key and cert, again then there's a chicken/egg problem with the password for the PKCS12 file, but again it can use like something above (in process pull with Vault, AWS, Azure, GCP etc)

Notes

• Yes there's auto_encrypt that could reduce usage, but there's still all the consul servers as well as it's super consul connect centric which isn't everyone's use cases

Issues

I really don't mind trying to take or work with someone and create a PR with this, but want to make sure there's standards and it's accepted by Hashicorp.

• Would this want to be done in core code?
• Mutually exclusive flags/config values between encrypted and non encrypted values would be needed
• Would this want to be done via a Consul Integration (if it would make sense to do it here)?
• Secrets/PKI/etc tend to have different flavors and smells and just want to make sure they fit the hashicorp culture/smells

[plainee]

It is expected that consul can provide a method for reading the private key password from memory or ENV variables. Or consul can interconnect with some KMS providers to decrypt the encrypted TLS key.