Propose structure of Conjur configuration ConfigMap to be used by Kubernetes applications

[izgeri]

Here are some of the common variables that all applications within a namespace would need to know:

• Kubernetes authenticator ID (AUTHENTICATOR_ID)

• Conjur account (CONJUR_ACCOUNT)

• Kubernetes service name for cluster’s Conjur instance (CONJUR_SERVICE_NAME)

  • Is this needed for follower outside of Kubernetes?

• Kubernetes service account name for cluster’s Conjur instance (CONJUR_SERVICE_ACCOUNT_NAME)

  • Is this needed for follower outside of Kubernetes?
  • Kubernetes namespace for cluster’s Conjur instance (CONJUR_NAMESPACE)
  • Is this needed for follower outside of Kubernetes?

• Conjur instance ClusterRole name (CONJUR_CLUSTERROLE_NAME)

• Conjur instance URL (CONJUR_APPLIANCE_URL)

  • For in-cluster Conjur instance, the value of this should just be https://${CONJUR_SERVICE_NAME}.${CONJUR_NAMESPACE}.svc.cluster.local

• If you review Authn-K8s Client, Secrets Provider, and Secretless, does this list include all variables that would be needed to configure the Conjur connection in the app manifest? Are any variables missing?

• If all of these are available in the ConfigMap, what would be left to configure in the app manifest? Can you share an example of what the app manifest would look like if these were in a ConfigMap?

• How many of these are cluster-wide configuration values vs workload / app specific?

• Does Kubernetes support cluster-wide ConfigMaps to avoid duplicating configuration? If we can't use a single ConfigMap for all apps, what options are there for standardizing / protecting the configuration? (eg standardized instructions for copying the ConfigMap, immutable ConfigMaps, etc)

If we make this configuration available in an ConfigMap that is created when the follower is deployed in the cluster, and we update the authenticator client logic to be able to read Conjur configuration from this ConfigMap, then the application manifests would no longer have to include references to multiple Conjur variables and would instead just need to know the location of the ConfigMap.

[izgeri]

@diverdane Alex just reminded me that we have a standard for configuration files for Conjur

• at a minimum it's documented here - conjur.conf, conjur.identity
• we have an example in Puppet here - conjur.conf, conjur.identity
• the ruby client also talks about .netrc for identity
• the cli also talks about .conjurrc for configuration

I might work on making a dap-wiki page that covers all four iterations and what clients support which config files, but I wanted to flag this for you as soon as I was reminded of it

[diverdane]

I looked at the configuration that each authn-k8s integration is using:

					authn-k8s Sidecar	+ Summon	authn-k8s init	+ Summon
Variable	Description	Examples	Cluster-wide signficance	Secretless Broker	authn-k8s sidecar	Summon	authn-k8s init	Summon	Secrets Provider Init	Secrets Provider App Container
CONJUR_AUTHN_URL	URL for the Kubernetes authenticator with which clients authenticate	https://conjur-oss.conjur-oss.svc.cluster.local/authn-k8s/my-authenticator-id	Yes	*	*		*		*	*
CONJUR_APPLIANCE_URL	Conjur HTTPS endpoint	https://conjur.myorg.com https://conjur-oss.conjur-oss.svc.cluster.local	Yes	*		*		*	*	*
CONJUR_ACCOUNT	Account specified during Conjur setup		Yes	*	*	*	*	*	*	*
CONJUR_AUTHN_LOGIN	Application (host) identity	host/production/redis001		*	*		*		*	*
CONJUR_SSL_CERTIFICATE	Public Conjur SSL Cert		Yes	*	*	*	*	*	*	*
CONJUR_AUTHN_TOKEN_FILE	File containing Conjur JSON access token,Base64 encoded					*		*
CONJUR_AUTHN_API_KEY	Appliation (host) API key	sb0ncv1yj9c4w2e9pb1a2s"
CONJUR_POLICY_ID	Policy namespace (for variables)	redis-v1
CONTAINER_MODE	Mode of operation for auth client container	sidecar, init, application			*		*		*	*
K8S_SECRETS	Kubernetes secrets which applications will use to access their secrets values.	my-k8s-secret-1, my-k8s-secret-2							*	*
SECRETS_DESTINATION	Destination for secrets retrieved by Secrets Provider	k8s-secrets							*	*
DEBUG		true							*	*
RETRY_INTERVAL_SEC	Sets interval for retrying retrieval of secrets	1
RETRY_COUNT_LIMIT		5

[diverdane]

Some of the other variables mentioned in the description appear to be used as intermediate values that are used to generate supporting Kubernetes resources, e.g.:

• AUTHENTICATOR_ID
• CONJUR_SERVICE_NAME
• CONJUR_SERVICE_ACCOUNT_NAME (Used to create a RoleBinding for providing the authn-k8s authenticator access to resources in the application namespace)
• CONJUR_CLUSTERROLE_NAME (Same here, used to create a RoleBinding in the application namespace)
• CONJUR_NAMESPACE (Used for creating a CLI pod in the Conjur namespace???)

Kubernetes ConfigMaps are namespaced resources, so there's no way to create one with cluster-wide scope. (BTW, to see all namespaced resources, use: kubectl api-resources --namespaced=true

[izgeri]

it's worth noting that CONJUR_AUTHN_URL can be constructed from CONJUR_APPLIANCE_URL and the AUTHENTICATOR_ID - I'd suggest that we expose the AUTHENTICATOR_ID as a config param vs the authn url.

See for example https://github.com/conjurdemos/kubernetes-conjur-demo/blob/9476aa63c9308aba219604caf53699816d7e175f/7_app_deploy.sh#L74

CONJUR_NAMESPACE is used to write the CONJUR_APPLIANCE_URL using the internal K8s service mapping

[diverdane]

Here's what I propose:

We define 2 ConfigMaps that will get populated in every Conjur-enabled application namespace. How these ConfigMaps get created and populated is discussed below. The ConfigMaps:

• One ConfigMap containing common (cluster-wide) Conjur connection information. This ConfigMap would be duplicated in every Conjur-enabled application namespace. Here is my best guess at the fields that this common ConfigMap should contain:

  • AUTHENTICATOR_ID: Used for creating CONJUR_AUTHN_URL and for checking that authn-k8s is enabled
  • CONJUR_APPLIANCE_URL: Used by authn-k8s, Summon, secrets provider, API clients
  • CONJUR_ACCOUNT: Used by authn-k8s, Summon, secrets provider, API client
  • CONJUR_SSL_CERTIFICATE
  • CONJUR_SERVICE_ACCOUNT_NAME (used to create a namespaced RoleBinding for authn-k8s)
  • CONJUR_CLUSTERROLE_NAME (used to create a namespaced RoleBinding for authn-k8s)
  • CONJUR_NAMESPACE (used to create a namespaced RoleBinding for authn-k8s)
  • PLATFORM (kubernetes or openshift)

  NOTE: bolded fields are those for which some automation might be useful for populating, i.e. values that might be "learned" by introspection of the cluster, as discussed below.

• A second ConfigMap containing application-specific Conjur Configuration, e.g.:

  • CONJUR_AUTHN_LOGIN
  • CONJUR_AUTHN_TOKEN_FILE
  • CONTAINER_MODE (e.g. sidecar, init, or app-container)
  • K8S_SECRETS (for secrets provider, list of secrets used by app)
  • SECRETS_DESTINATION
  • DEBUG
  • RETRY_INTERVAL_SEC
  • RETRY_COUNT_LIMIT

Some Notes / Limitations:

• ConfigMaps are scoped to a namespace, so we can't define one "global" ConfigMap, let's say in the Conjur namespace, or in a sidecar injector namespace, unless we give all applications access to all ConfigMaps in that Conjur/Injector namespace... which is not a good idea.
• How do we initially populate the common ConfigMap? Most fields are fairly basic/primitive and would be easy to supply e.g. as values to a Helm chart... but there are others that could use some automation to retrieve from a cluster (to make it easier for cluster operators to deploy namespaces and to avoid misconfiguration). This might include:
  • CONJUR_SSL_CERTIFICATE
  • CONJUR_SERVICE_ACCOUNT_NAME
  • CONJUR_CLUSTERROLE_NAME
• We can't just add endpoints to the Conjur server to help in retrieving these values from the cluster, since the server isn't aware of things like
  • What it's SSL certificate is (that's configured in NGINX).
  • What Kubernetes service account it is using
  • What the name of the Conjur ClusterRole is
• The Conjur admin password obviously can't be stored in a ConfigMap due to its sensitive nature. This will have to be retrieved/rotated each time it is required (e.g. for loading Conjur policy).

So how would the common ConfigMap get populated? I think some of the fields are primitive information that the cluster operator would need to supply (i.e. can't be learned via introspection of the cluster):

• AUTHENTICATOR_ID
  • CONJUR_APPLIANCE_URL
  • CONJUR_ACCOUNT
  • CONJUR_NAMESPACE
  • PLATFORM

But for these other fields, it may be possible to develop automation to retrieve these from the cluster:

• CONJUR_SSL_CERTIFICATE
• CONJUR_SERVICE_ACCOUNT_NAME
• CONJUR_CLUSTERROLE_NAME

For these fields, I can see a few options for providing automation:

(1) ConfigMap Generator CLI: Create a new CLI tool that can gather information from the cluster. Generated output could either be a YAML manifest, or an actual ConfigMap resource.

(2) "Helm Chart Helper" CLI: Create a new CLI tool that can gather information from the cluster (similar to (1)). However, the output from the "Helm Chart Helper" CLI would be chart values for a namespace preparation Helm chart. The process would be:

• Run the Helm chart helper CLI to generate values
• Run 'helm install ...' for application namespace prep using those values

(3) App Namespace Prep Operator, with App Namespace CRD (highest level of automation) All of the logic of gathering info from the cluster would be built into an App Namespace Prep operator (instead of building this logic into an external CLI tool as in (1) and (2)). The App Namespace Prep operator would be running in its own namespace, and would be continuously monitoring for creation of a new "App Namespace" Custom Resource. WHen a new "App Namespace" CR is detected, the operator would gather all of the required Conjur config (it may need to incorporate the Kubernetes API to access this info) and would create an Application namespace based on info from the CR and info learned via introspection of the cluster.

I think (2) is the most reasonable approach.

[izgeri]

@diverdane today I realized that the sidecar injector already defines configmaps for Conjur config and Conjur auth config: https://github.com/cyberark/sidecar-injector#sidecar-injectorcyberarkcomconjurconnconfig