vmware-tanzu / secrets-manager

VMware Secrets Manager is a lightweight secrets manager to protect your sensitive data. It’s perfect for edge deployments where energy and footprint requirements are strict—See more: https://vsecm.com/
https://vsecm.com/
BSD 2-Clause "Simplified" License
134 stars 24 forks source link

VSecM high-trust mode: Use a PKCS#11 interface to secure VSecM Root Keys #731

Open v0lkan opened 3 months ago

v0lkan commented 3 months ago

VSecM high-trust mode: Use a PKCS#11 interface to store VSecM root keys in a hardware security module and keep secrets encrypted (even in memory). When a workload requires a secret VSecM safe will use HSM/PKCS#11 to decrpyt the secret and send it to the workload via mTLS that is protected by SPIRE — in addition SPIRE root keys shall also be stored in HSM.

This will protect VSecM against advanced attack vectors such as…

Root Users and Kernel Mode: Processes with root privileges, or kernel mode drivers, can access or manipulate the memory of other processes.

Memory Dump Risks: Under certain conditions (like a system crash), the content of the memory can be dumped to disk, potentially exposing sensitive information.

Side-Channel Attacks: Sophisticated attacks may infer the values of secrets through patterns in memory access, cache usage, or other indirect means.

v0lkan commented 3 months ago

An HSM is a dedicated crypto processor designed to securely manage and store cryptographic keys and perform encryption operations. It offers a higher level of security against physical and logical attacks compared to general-purpose computing environments. Using an HSM for encrypting secrets in memory would theoretically provide the following benefits:

Enhanced Security: By offloading the encryption and decryption operations to an HSM, the cryptographic keys can be kept out of the host system's memory entirely, reducing their exposure. Physical Tamper Resistance: HSMs are designed to resist physical tampering and can automatically zeroize sensitive information if a breach is detected. Compliance and Auditability: For certain regulatory environments, using an HSM to handle encryption keys can help in meeting compliance requirements regarding key management.

v0lkan commented 3 months ago

Note that this will create a performance overhead, and shall be considered in highly-sensitive, highly-paranoid, and highly-regulated environments only.

v0lkan commented 3 months ago

I thought a bit more on this.

We’ll need several “drivers” for this, and we probably need to split this to sub-stories as this is more an "epic" than a story.

Step 1: Anything in VSecM Safe’s Memory shall be encrypted.

  1. Create a private/public key pair (PriK, PubK)
  2. Store PriK as a Kubernetes Secret
  3. Store PubK on disk (PV) and in memory
  4. During bootstrap, create the root key as usual, then encrypt it with PubK, store it in memory and the root key secret in encrypted form ( Enc(RK) )
  5. when registering a secret, first decrypt the root key using the private key ( Enc(RK) -> RK ) then use the root key to encrypt the secret, and then store the secret in encrypted form ( Enc(S) ) in FileSystem and memory.
  6. When sending a secret to a workload, decrypt the root key, decrypt the secret using the decrypted root key, and send the secret to the workload that way.

All of this so far is PHASE 1. This will NOT be the default behavior; the user must opt in to use this enhanced security because it will come with an additional computational overhead.

Here’s Phase 2:

  1. Create the key pair but store the private key in an HSM instead of a Kubernetes secret.
  2. Provide the public key using a ConfigMap. The key pair is created off-cyle; it’s not part of the bootstrap process (simply because you cannot send the private key to HSM without using a privileged container, so it’s better to do this off-cycle as it will be more secure).
  3. Create a binary that uses PKCS#11 interface to decrypt things using the private key stored in the HSM.
  4. call the binary via SSH and get whatever result you need.

For 4, you’ll provide the public key (of the pod) to the host machine; and you’ll need to secure the private key.

This can be done in an init container. The init container can dynamically generate the key pair, and share it in tmpfs.

Also, since a new key pair will be created whenever the pod is restarted, host machine will need to know this to allow SSH access.

That can be done by storing the public key in a db, a trusted script watching the db for changes, and updating authorized_keys depending on the change in the db.

All this is to make this work in a bare-metal on-premises HSM.

Phase 3: Cloud HSMs.

For cloud HSMs, we can leverage their API instead of using init containers, and executing binaries via SSH.