Proposal: Extension for Declarative Kubernetes Credential Scoping

[jlegrone]

Background

285 has raised important points about the need to avoid granting overly permissive authority to invocation images via CNAB credentials.

The spec does not currently describe a mechanism for allowing bundles to communicate to a runtime what permissions they require in order to deploy to the underlying platform target. This problem is not specific to Kubernetes, but because Kubernetes deployments are a major use case for early adopters of CNAB, this is a useful context for discussing possible solutions.

Proposal

Objectives

The goal is to add a non-normative extension to the spec which

• Allows runtimes to authorize Kubernetes API scopes requested by bundles
• Is agnostic to the underlying application definition, rendering, or deployment mechanism

We can follow a similar pattern to that laid out by well-known custom actions to use well-known CNAB parameter and credential names, along with a custom extension for configuring RBAC, to achieve these objectives.

Design

1. Enable bundles to specify required Kubernetes API permissions

Bundles that deploy Kubernetes applications SHOULD implement the Kubernetes RBAC custom extension. The extension would allow bundles to specify the following attributes:

• rules (required) same data as that defined in the rbac.authorization.k8s.io/v1 Role API
• namespaces (optional) list of namespaces in which rules should be bound

Example:

{
    "custom": {
        "io.kubernetes.rbac": {
            "version": "v1alpha1",
            "rules": [
                {
                    "apiGroups": [""],
                    "resources": ["pods", "pods/log"],
                    "verbs": ["get", "watch", "list"]
                },
                {
                    "apiGroups": ["apps"],
                    "resources": ["daemonsets"],
                    "verbs": ["get", "watch", "create", "update", "patch", "delete"]
                }
            ],
            "namespaces": ["logs-agent"]
        }
    }
}

Furthermore, if bundles support deploying into arbitrary Kubernetes namespaces, they should use the CNAB parameter io.kubernetes.namespace. When this parameter is present, its value is appended to the list of namespaces specified in the Kubernetes RBAC extension.

If no namespace parameter is present and the namespaces attribute of the extension is empty, rules are assumed to be cluster scoped.

2. Allow runtimes to identify which CNAB credentials are Kubernetes credentials

Bundles that deploy Kubernetes applications SHOULD declare a credential named io.kubernetes.kubeconfig.

If both the io.kubernetes.rbac extension and a io.kubernetes.kubeconfig credential are present in a bundle, a CNAB runtime that is aware of the Kubernetes extension would be able to construct a kubeconfig with appropriately scoped permissions and pass this to the invocation image. When running actions that do not have "modifies": true set, the runtime would also be free to remove verbs like create, update, patch, and delete from the provided scopes.

Other Considerations

This proposal is almost certainly incomplete; there is not enough expressivity in the extension API to account for deployments that need different permissions in different namespaces, or that deploy to multiple dynamically configured namespaces. There is also no way to specify different permissions between install, upgrade, delete, or other custom actions beyond what the runtime can infer from the modifies action attribute.

This proposal also makes no attempt to define a standard Kubernetes packaging and deployment specification as requested in #285, but neither would it conflict with such a specification.

[technosophos]

My understanding is that you are declaring a new custom extension that would allow embedding k8s RBACs, right? It's a little confusing when you way:

We can follow a similar pattern to that laid out by well-known custom actions to use well-known CNAB parameter and credential names, along with a custom extension for configuring RBAC, to achieve these objectives.

But the data structure you use looks like a custom extension.

That sounds like a great idea to me. As one of the non-required extensions, this would make it easy for Kubernetes users to make use of this without impacting the non-Kubernetes cases.

I think it could also give us a reasonable pattern to suggest for other agents that want to embed authz data.

[jlegrone]

But the data structure you use looks like a custom extension.

Yes, this issue describes a custom extension. So far I haven't seen any extensions that also leverage naming conventions for credentials or parameters, and this proposal does both. I referenced "well known" custom action names to demonstrate that this is an established pattern for other areas of the spec.

[technosophos]

Excellent! I am in favor of this proposal.

[carolynvs]

I asked a few questions about this at last week's meeting and wanted to copy some of the answers here (from the meeting notes) just so that people can see them:

• Carolyn: Is idea that bundle says these are the credentials the bundle needs and that the expectation is that the runtime creates user/service config with these?
  • Jacob: Idea is to pass admin credentials and have them down privileged. Unfortunately this isn't practically easy to do
• Carolyn: Where would the code that does this live, CNAB Go?
  • Matt: This would be non-normative so could be implemented in an external tool
  • Jacob: For CNAB Go, we could have an interface for multiple credential providers and that spits out credential sets
  • Carolyn: So this would be performed before the bundle is executed so the injected credential is already transformed.
• Carolyn: Wondering how portable bundles like this would be
  • Jacob: Goal is to have something portable and avoid having bundles only be able to be installed by one tool
  • Carolyn: Tool that understands this extension has better security, naive tool still works