Wildcard verbs causes problems on OpenShift

[git001]

We try to install at kots app and receive always the same error.

failed to update role: roles.rbac.authorization.k8s.io "kotsadm-role" is forbidden: \
user "$USER" (groups=["$GROUP" "system:authenticated:oauth" "system:authenticated"]) \
is attempting to grant RBAC permissions not currently held:

{APIGroups:[""], Resources:["configmaps"], Verbs:["*"]}
{APIGroups:[""], Resources:["secrets"], Verbs:["*"]}

The user have admin permissions on the namespace but not cluster-admin rights.

It looks to me that this part is the reason for this error.

https://github.com/replicatedhq/kots/blob/1307dd684bfacd2e0d259a67842b22629a7a8eb2/pkg/kotsadm/kotsadm_objects.go#L45

The return value of this command is no. kubectl -n $namespace auth can-i '*' 'configmap'

But this is command returns yes. kubectl -n $namespace auth can-i 'create' 'configmap'

How about to add dedicated permissions to the policy instead of "*"?

[git001]

close as it looks stale

[calderonth]

Can this be re-opened?

I think there is still an issue with the RBAC used by kotsadm. For instance when deploying using --use-minimal-rbac kots still expects pretty much cluster-admin in the namespac which is problematic.

I was able to successfully install a kots application by patching the KotsadmRole to a dummy role and modifyin KotsadmRoleBinding function to use ClusterRole instead of Role, mapped to admin.

For instance:

diff --git a/pkg/kotsadm/objects/kotsadm_objects.go b/pkg/kotsadm/objects/kotsadm_objects.go
index b64995222..abbea6f92 100644
--- a/pkg/kotsadm/objects/kotsadm_objects.go
+++ b/pkg/kotsadm/objects/kotsadm_objects.go
@@ -54,9 +54,9 @@ func KotsadmRole(namespace string) *rbacv1.Role {
        },
        Rules: []rbacv1.PolicyRule{
            {
-               APIGroups: []string{"*"},
-               Resources: []string{"*"},
-               Verbs:     metav1.Verbs{"*"},
+               APIGroups: []string{""},
+               Resources: []string{"pods"},
+               Verbs:     metav1.Verbs{"list"},
            },
        },
    }
@@ -111,8 +111,8 @@ func KotsadmRoleBinding(roleBindingNamespace string, kotsadmNamespace string) *r
        },
        RoleRef: rbacv1.RoleRef{
            APIGroup: "[rbac.authorization.k8s.io](http://rbac.authorization.k8s.io/)",
-           Kind:     "Role",
-           Name:     "kotsadm-role",
+           Kind:     "ClusterRole",
+           Name:     "admin",
        },
    }

Is there a reason why this isn't the approach taken when attempting to install in a namespace with minimal RBAC permissions?

Thanks

[divolgin]

@calderonth currently the only workaround is to specify the --ensure-rbac=false flag on the command line and to create the role object out of band with all the allowed permissions.

[calderonth]

OK I can try that but it doesn't answer why the wildcard verbs are used in the first place.

For the namespace deployment using a ClusterRole mapped to admin was enough (on GKE). kots should attempt to do least priviledge instead of the wildcard permissions.

Is there other reasons I am missing?

[divolgin]

This is done because resources can be added to cluster dynamically, and there is no way to know them ahead of time.

Kots also won't default to user's permissions because they can be insufficient (either now or in the future). So expectation is that the installation is performed by someone with sufficient permissions or Role/ClusterRole is tailored to the specific cluster's needs.

[calderonth]

Is there a known example where admin in a namespce would not be sufficient for kots to operate?

[divolgin]

Yes, examples are easy to create. There is one in your diff. This will prevent kotsadm from deploying an application for example.

-               APIGroups: []string{"*"},
-               Resources: []string{"*"},
-               Verbs:     metav1.Verbs{"*"},
+               APIGroups: []string{""},
+               Resources: []string{"pods"},
+               Verbs:     metav1.Verbs{"list"},

[calderonth]

Yes, examples are easy to create. There is one in your diff. This will prevent kotsadm from deploying an application for example.

-             APIGroups: []string{"*"},
-             Resources: []string{"*"},
-             Verbs:     metav1.Verbs{"*"},
+             APIGroups: []string{""},
+             Resources: []string{"pods"},
+             Verbs:     metav1.Verbs{"list"},

I don't think that's quite right, I've purposedly made the kotsadm-role a dummy one but instead of referencing to it in the role binding, I instead map to the ClusterRole kind mapped to admin. Allowing to deploy successfully an application.

[divolgin]

Correct, it's not right. That's an example of what doesn't work. Any definition is not the right one. The list of permissions is dynamic.