search

minio / kes

Key Managament Server for Object Storage and more
https://min.io/docs/kes/concepts/
GNU Affero General Public License v3.0
456 stars 95 forks source link

WIP: add stateful KES servers and distributed clusters #394

Closed aead closed 8 months ago

aead commented 1 year ago

This commit introduces the initial implementation of a stateful KES server.

Now, there are two KES server variants:

The previous edge server remains unchanged, and this commit does not impact the behavior of existing edge server deployments, even though its implementation has evolved. However, note that the command to start an edge server has been renamed from kes server to kes edge. The kes server command is now used for the stateful KES server. This is considered a breaking change.

The stateful KES server, henceforth referred to as "KES server," implements a superset of the edge server API. Applications that use the KES edge server API should also be compatible with the KES server API.

Clusters

A KES cluster can consist of any number of servers. A single server represents the smallest cluster with just one node. Clusters can dynamically expand or shrink as servers join or leave. All server nodes within a cluster participate in a consensus algorithm inspired by Raft (read more: Raft Consensus Algorithm).

However, a KES cluster offers different availability and fault tolerance guarantees and provides strict consistency without incurring performance costs. Specifically:

  1. A KES cluster can make progress (process write requests) only when there is a leader and all N nodes are available. In Raft, a cluster can make progress as long as there is a leader and a majority (N/2 + 1) of the nodes are available.
  2. A KES cluster ensures strict consistency as long as at least one node is available. In Raft, strict consistency relies on having a leader and a majority (N/2 + 1) of the nodes available, with additional performance trade-offs.

Each KES server maintains a local database containing server state (keys, policies, identities, secrets, etc.) and a cluster configuration. This database is stored in a single file, kes.db, within the KES server's working directory. The cluster configuration is kept in the .cluster.json file, also located in the working directory.

API

The github.com/minio/kes repository now exposes public APIs for utilizing KES servers and KES edge servers in third-party applications. These package APIs are inspired by the Go net/http package:

KES servers, as implemented by the github.com/minio/kes/cmd/kes main package, accept a configuration file. Reading and parsing such configuration files is facilitated by the kes/kesconf and kes/edge/edgeconf packages.

Security

KES edge servers employ a central Key Management Service (KMS) as a persistent key store. The entire edge server state is either ephemeral or provided by the configuration file.

Stateful KES servers maintain local persistent state in a single-file key-value database (Bolt DB). These KES servers encrypt their database using a root encryption key, which is also stored within the database. The question arises: How is the root encryption key protected when it is stored in the database?

The answer lies in Hardware Security Modules (HSMs). KES servers can leverage HSMs to seal and unseal their root encryption keys. Any specific HSM can be supported if there is an implementation of the kes.HSM interface type, which abstracts various HSM implementations including platform TPMs, cloud HSMs (AWS, GCP, etc.), and HSM manufacturers. Currently, KES ships with a software implementation of an HSM via the kes.SoftHSM type, simplifying deployment without specific hardware requirements and significantly streamlining the setup process.

aead commented 1 year ago

@ravindk89 Adding you for reviewing some of the documentation. Many parts are currently missing.

@shtripat While this PR is marked as WIP (mainly due to fixing lints, docs and adding tests and examples) it should be reviewable. (I'm sry about the size).

I will add separate commits to this PR and squash them once approved. This should help at least a bit for reviewing. I expect new changes to be mostly docs, test and minor improvements / addressing review comments.

harshavardhana commented 8 months ago

I am closing this PR as stale and outdated.