Open ChristianRaoulis opened 6 months ago
blampe
commented
6 months ago Hi @ChristianRaoulis, this is a known issue with the v3 Chart resource, specifically https://github.com/pulumi/pulumi-kubernetes/issues/2227.
We're currently working on a v4 Chart which will address this and many other issues https://github.com/pulumi/pulumi-kubernetes/issues/2847 -- please stay tuned!
EronWright
commented
6 months ago I would suggest using Release resource as a workaround for now. It is effective because previews of a Release doesn't perform a dry-run.
Even if roles were created before bindings, the role wouldn't actually exist during preview and so the preview logic would need to be special-cased. Typically, Kubernetes accepts RoleBinding objects that refer to a non-existent Role. But I think I found an explanation here, it is that @ChristianRaoulis perhaps doesn't have admin access to the cluster. Could you confirm?
Update: I am able to repro the issue when I use a non-admin account. Here's how:
# admin.yaml - grant 'admin' role to 'myuser' in 'default' namespace, to be able to create bindings.
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: myuser-default-admin
namespace: default
subjects:
- kind: User
name: myuser
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: admin
apiGroup: rbac.authorization.k8s.io# rabbitmq.yaml - grant 'foobar' role (non-existent) to 'rabbitmq' in 'default' namespace.
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: rabbitmq-foobar
namespace: default
subjects:
- kind: ServiceAccount
name: rabbitmq
namespace: default
roleRef:
kind: ClusterRole
name: foobar
apiGroup: rbac.authorization.k8s.io❯ kubectl apply -f admin.yaml
rolebinding.rbac.authorization.k8s.io/myuser-default-admin created
❯ kubectl apply -f rabbitmq.yaml --as myuser
Error from server (NotFound): error when creating "rabbitmq.yaml": clusterroles.rbac.authorization.k8s.io "foobar" not found
❯ kubectl apply -f rabbitmq.yaml
rolebinding.rbac.authorization.k8s.io/rabbitmq-foobar createdI see two possible improvements here:
EronWright
commented
6 months ago @ChristianRaoulis I notice that the resource URN has an unusual value:
urn:pulumi:testing::softwarefactory::kubernetes:core/v1:Namespace$softwarefactory:rabbitmq$kubernetes:helm.sh/v3:Chart$kubernetes:rbac.authorization.k8s.io/v1:RoleBinding::core-services/rabbitmq-endpoint-readerIt seems like a core-services namespace resource is involved, and maybe being used as a parent? Please use dependsOn rather than parent. It is a separate problem but I felt I should mention it.
ChristianRaoulis
commented
6 months ago Hi @ChristianRaoulis, this is a known issue with the v3 Chart resource, specifically #2227.
We're currently working on a v4 Chart which will address this and many other issues #2847 -- please stay tuned!
Is there a time horizon when Chart V4 will be usable? Days, weeks, months?
ChristianRaoulis
commented
6 months ago I would suggest using
Releaseresource as a workaround for now. It is effective because previews of aReleasedoesn't perform a dry-run.
Thank you :)
Even if roles were created before bindings, the role wouldn't actually exist during preview and so the preview logic would need to be special-cased. Typically, Kubernetes accepts
RoleBindingobjects that refer to a non-existentRole. But I think I found an explanation here, it is that @ChristianRaoulis perhaps doesn't have admin access to the cluster. Could you confirm?
Confirmed 👍
It seems like a
core-servicesnamespace resource is involved, and maybe being used as a parent? Please usedependsOnrather thanparent.
I passed the namespace to the chart using the namespace: namespace.metadata.name prop but the result is the same :)
blampe
commented
5 months ago @ChristianRaoulis the new Chart v4 resource is ready for beta testing if you'd like to try it out! Feel free to reach out via email (in my GH profile) or in the Pulumi community Slack if that's something you're interested in.
EronWright
commented
5 months ago I do not think that Chart v4 will solve this issue, because the problem isn't related to ordering, it is due to the special case behavior of RoleBinding when you're a non-admin. The only real fix would be to add some special-case logic to the provider, e.g. to use client-side validation only for RoleBinding during preview, or to swallow the error.
ChristianRaoulis
commented
4 months ago I do not think that Chart v4 will solve this issue, because the problem isn't related to ordering, it is due to the special case behavior of
RoleBindingwhen you're a non-admin. The only real fix would be to add some special-case logic to the provider, e.g. to use client-side validation only forRoleBindingduring preview, or to swallow the error.
I can confirm that.
I ran into the role not found error again but this time with a role that i create myself in the same preview.
const role = new Role('role', {
metadata: {
name: 'service-account-role',
namespace: namespace.metadata.name,
},
rules: [
{
apiGroups: [""],
resources: [
"secrets",
],
verbs: [
"get",
"update",
],
},
],
});
const serviceAccount = new ServiceAccount("service-account", {
metadata: {
name: "service1",
namespace: namespace.metadata.name,
},
});
const roleBinding = new RoleBinding('role-binding', {
metadata: {
name: 'role-binding',
namespace: namespace.metadata.name,
},
roleRef: {
apiGroup: "rbac.authorization.k8s.io",
kind: "Role",
name: role.metadata.name,
},
subjects: [
{
kind: "ServiceAccount",
name: serviceAccount.metadata.name,
},
],
}, {parent: serviceAccount});Is there any workaround i could use for now?
What happened?
I'm trying to deploy the RabbitMQ Helm Chart of Bitnami using the
helm.v3.Chartclass but for some reason pulumi tried to create aRoleBindingbefore the isRolecreated.Relevant log from
pulumi preview:Example
Output of
pulumi aboutAdditional context
No response
Contributing
Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).