#noperproductconsulacls - Secrets for all tiers of each product are available to instances in all tiers of that product

[tinnightcap]

Consul's method of controlling access to the secrets which it stores is by requiring that clients requesting secrets authenticate themselves with a x509 keypair signed request. The request must be signed by a x509 keypair which has been signed by a predefined certificate authority. The consul server has the public key of this certificate authority set so that only clients with certs signed by that CA can request secrets.

All consul clients use the came client certificate
The client certificate is self-signed
Clients obtain the client certificate by fetching it from a unauthenticated internet accessible http endpoint
https://github.com/Nubisproject/nubis-base/blob/9faad0ff20e4964579bea09b5c7737e9426a753f/nubis/bin/consul-autojoin.sh#L61-L62
The result of this is that any instance in a given product AWS account can fetch all secrets for all tiers of that product. For example a production webserver for Product A can obtain secrets for a production backend DB for Product A which are only needed by a production middleware tier.

This design doesn't allow for differing security levels within a product. Since all hosts have access to all secrets for the product, the weakest link (perhaps with the greatest attack surface because it's a webserver) is the maximum security all that product's instances can obtain. Because staging and production are in different VPCs and security groups control access to the Consul API, staging instances can not access production secrets.

A planned mitigation for this issue is to enable jenkins to provision an ACL token from Consul and inject it into a stack via CloudFormation and then into the ec2 instance via user_data

https://github.com/Nubisproject/nubis-consul/issues/27#issuecomment-119276929
https://github.com/Nubisproject/nubis-consul/issues/27#issuecomment-122999264

This security control will reduce this risk but it will remain significant. In a model with this security control, the ACL token and by extension all secrets which that instance has access to (or all secrets if there is no ACL differentiation) are available to all users with the ability to read CloudFormation stack outputs or to read instance user_data .

[tinnightcap]

The solution is to properly implement ACLs. That work is being tracked in: https://github.com/nubisproject/nubis-consul/issues/85

[gozer]

Since the work in #138, the platform secrets are segmented out from the application secrets. It's not a complete solution, but it's a start.

[gene1wood]

The remaining tasks to review this are :

• Determine if #138 prevents app instances from accessing nubis infrastructure secrets, prevents nubis infrastructure instances from accessing app secrets or both
• Assess what if any change to the #noperproductacls risk level there is with the #138 changes which
  • continues to permit nubis infrastructural instance A (for example a fluent collector) to access nubis infrastructural instance B (for example a bastion server) consul secrets
  • continues to permit app instance A (for example a web server) to access app instance B (for example a middleware server) consul secrets
  • prevent app instance A (for example a web server) from accessing nubis infrastructural instance B (for example a bastion server) consul secrets
  • prevent nubis infrastructural instance A (for example a fluent collector) from accessing app instance B (for example a middleware server) consul secrets
• Validate that #142 does actually cause these access control constraints

A mitigating factor to this is that JD reports that current products which use nubis don't have multiple tiers (meaning there are no current products using nubis with different types of instances which use different secrets, like a webserver and db server)