ADR for microservice communication security

[bnevis-i]

🚀 Feature Request

Description

The EdgeX framework currently relies on network segmentation for security. An API gateway separates external traffic from internal traffic, authenticates callers, and forwards properly authenticated requests to internal services. Services behind the gateway share a common network, and communication between microservices is unrestricted.

The EdgeX framework should do a better job of protecting itself against local attacks such as unauthorized software processes running on the host or malicious containers. The secret management subsystem of EdgeX is the current sole exception to the unrestricted communication posture:

• The secret store must enforce role-based access control policies.
• The secret store must protect the confidentiality of secrets in transit.
• The secret store must be available for proper system functioning.

Besides secret management, there are other sensitive services in EdgeX, such as the systems management agent, that execute with a high level of privilege and expose control-plane level functionality. These other microservices also require confidentiality and integrity protections on data, as well as role-based access controls that require the caller to authenticate itself. Moreover, not all microservice communications are many-to-many; for example, the only microservice that needs to access the PostgresSQL database is the API gateway.

Describe the solution you'd like

This feature request is a placeholder to begin discussion of:

• Implementation of TLS security between microservices to provide confidentiality and integrity protection of messages.
• Implementation of microservice-to-microservice authentication.
• Implementation of a common authorization framework for microservice implementers.
• Implementation of policy-based networking.

The solution needs to comprehend:

• Security-enabled vs non-security enabled EdgeX stack
• Authentication/authorization of requests proxied through the API gateway
• Authentication/authorization of internal service-to-service requests

Describe the solution space (hand-coded option)

Confidentiality and Integrity

Server-side TLS is the most convenient option here. The security-secrets-setup tool initializes PKI for Vault and could be extended to other services. Alternatively, security-secrets-setup can be modified to create an intermediate CA that is loaded into the secret store's PKI engine, and services can directly go there to get TLS certificates.

Microservice authentication

Mutual-auth TLS is one option here. This option combines confidentiality, integrity, and authentication together, but it imposes a tax: encryption is mandatory, which could add an unacceptable level of overhead for high-bandwidth or latency-sensitive applications.

Another option is token-based authentication. There are various options for selection of a token.

• Use the pre-existing secret store authentication token. The server could use this token to verify the client's identity with the secret store, but it could also steal all the client's secrets. Not a good solution
• Use the secret store token accessor. The server could use this token to query metadata about the caller out of the secret store. The benefit--and drawback--of this solution is that it allows for unlimited delegation: a far-removed microservice can perform actions on behalf of the original caller.
• Use a JWT token. The benefit of this approach is that use of an audience field can be used to prevent delegation of the token. The JWT could be signed by a JWT provider that everyone trusts, or a service could sign its own JWT. Self-signed JWT's require that the public key database be held in some kind of trusted registry (so that an attacker couldn't replace the key in the registry and then claim to be that microservice). Signed objects need to deal with TTL issues.

Microservice authorization

For the hand-coded option, this means that code must be added to each microservice to authorize each operation. This creates a high barrier for developers.

Network policy

Policy-based networking can be done to some extent with Docker networks. It is an effective even if authentication and authorization fail.

Describe the solution space (service-mesh option)

Many microservice architectures introduce a service mesh to deal with microservice encryption and authentication. Some popular service meshes include, but are not limited to:

• Consul Connect
• Linkderd
• Istio
• Envoy
• Conduit

Service meshes often solve the problems of confidentiality and integrity, authentication, authorization, and network policy at the same time. A network intermediary could combine authorization and network policy by blocking unauthorized requests. However, fine-grained authorization might require special support from the service mesh, such as by adding context to a request before it is delivered to a microservice.

Services known to be incapable of microservice communication security

• Redis. Redis supports only a single password and does not support TLS at all.

[andresrinivasan]

Redis 6 supports mutual auth TLS and RBAC. While I'm open to trying to upgrade for Geneva, I strongly recommend sticking with Redis 5.x for Geneva and switch to Redis 6 for Hanoi.

Redis 6 RC1 is out today

[bnevis-i]

Moved back to in-progress to make time to experiment with OpenZiti as a possible solution. Prototype committed in Levski planning 5/19/2022