Closed HSoulat closed 11 months ago
@HSoulat Have you read the RBAC part in the docs: https://cert-manager.io/docs/policy/approval/approver-policy/#configuration?
Are you able to share some details of your CertificateRequest, Issuer (or ClusterIssuer) and CertificateRequestPolicy?
@erikgb, sure, I may have missed something regarding the RBAC configuration
Issuer,
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: xxx-issuer
spec:
ca:
secretName: ca-xxx-key-pair
Policy,
apiVersion: policy.cert-manager.io/v1alpha1
kind: CertificateRequestPolicy
metadata:
name: xxx-policy
spec:
allowed:
...
isCA: false
usages:
- "server auth"
- "client auth"
- "key encipherment"
- "digital signature"
subject:
...
constraints:
...
selector:
# Select all IssuerRef
issuerRef: {}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: cert-manager-policy:xxx
rules:
- apiGroups: ["policy.cert-manager.io"]
resources: ["certificaterequestpolicies"]
verbs: ["use"]
# Name of the CertificateRequestPolicies to be used.
resourceNames: ["xxx-policy"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: cert-manager-policy:xxx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cert-manager-policy:xxx
subjects:
- kind: Group
name: system:authenticated
apiGroup: rbac.authorization.k8s.io
- kind: ServiceAccount
name: cert-manager
namespace: cert-manager
Certificate.
apiVersion: cert-manager.io/v1
kind: CertificateRequest
metadata:
annotations:
cert-manager.io/certificate-name: xxx-tls
cert-manager.io/certificate-revision: "1"
cert-manager.io/private-key-secret-name: xxx-tls-w4cdm
creationTimestamp: "2023-11-07T13:16:09Z"
generateName: xxx-tls-
generation: 1
name: xxx-tls-rwn9m
namespace: kube-system
ownerReferences:
- apiVersion: cert-manager.io/v1
blockOwnerDeletion: true
controller: true
kind: Certificate
name: xxx-tls
uid: ed749d1a-54cb-480e-a449-36bdfff232e1
resourceVersion: "23930574"
uid: c8808450-44f6-4655-a4db-1923bd0cac91
spec:
duration: 2760h0m0s
issuerRef:
group: cert-manager.io
kind: ClusterIssuer
name: xxx-issuer
request:
...
usages:
- digital signature
- key encipherment
I have also tested the same configuration with a selfsign issuer and I got the same behavior. The certificate requests in any namespace are approved execpted in kube-system
Definitely looks like there's something strange going on here!
That SubjectAccessReview (SAR) is created here. It uses the username from the certficaterequest.spec
but it doesn't look like there's a username in your CertificateRequest there.
Usually that should be set by the mutating webhook in cert-manager in the normal case and then checked by the cert-manager validating webhook. Are they working as expected for kube-system?
Building on what @SgtCoDFish said; perhaps the platform admin has modified the MutatingWebhookConfiguration
to avoid it being called for kube-system
objects:
Check for differences between the actual mutatingwebhook configuration (kubectl get mutatingwebhookconfiguration cert-manager-webhook -o yaml
) and the original cert-manager mutatingwebhookconfiguration.
What version of K8s? Which vendor? (GKE?)
Do you have access to API server logs? (For GKE, these have been quite illuminating)
@wallrj you're true, AKS platform is adding the following in namespaceSelector
on MutatingWebhookConfiguration
as well on ValidatingWebhookConfiguration
.
namespaceSelector:
matchExpressions:
- key: kubernetes.azure.com/managedby
operator: NotIn
values:
- aks
- key: control-plane
operator: NotIn
values:
- 'true'
kube-system namespace have both labels.
What is strange to me is that if I use the default approver in cert-manager the validation process is working.
What is strange to me is that if I use the default approver in cert-manager the validation process is working.
That's because the default approver in cert-manager always approves CertificateRequests. It ignores the empty identity fields in those kube-system CertificateRequest resources.
In a default installation, cert-manager automatically approves all CertificateRequests and CertificateSigningRequests that use any of its built-in issuers. This is done to simplify the first-time experience of using cert-manager. -- https://cert-manager.io/docs/policy/approval/
Thanks for your help to understand the root cause. So using a custom approver on AKS do not permit to generate CertificateRequests on kube-system or an option is available to allow build-in/empty user ?
Thanks for your help to understand the root cause. So using a custom approver on AKS do not permit to generate CertificateRequests on kube-system or an option is available to allow build-in/empty user ?
My advice is to treat kube-system
as the property of AKS and not to attempt to create or change any resources in that namespace.
But it looks like you can disable the "Admissions Enforcer" if you choose:
@HSoulat feel free to re-open this issue if necessary.
/close
@wallrj: Closing this issue.
Certificate approbation doesn't work with custom approval-policy on a certificate / certificate-request in
kube-system
namespace but work on others namespacesHere are the logs of approver policy pod :
On custon namespace
On kube-system
Cert manager v1.12.2 approver policy v0.8.0