search

cert-manager / istio-csr

istio-csr is an agent that allows for Istio workload and control plane components to be secured using cert-manager.
https://cert-manager.io/docs/usage/istio-csr/
Apache License 2.0
161 stars 71 forks source link

updating ConfigMap data doesn't stop #161

Open ceastman-r7 opened 2 years ago

ceastman-r7 commented 2 years ago

helm chart version:v0.4.2 for istio-csr

The log entry updating ConfigMap data just keeps repeating for all the namespaces in the cluster.

2022-06-13T16:23:06.577141Z info klog "msg"="-----------------------------------------------------------------"
2022-06-13T16:23:06.577166Z info klog "msg"="Using root CAs from file: /var/run/secrets/istio-csr/ca.crt"
2022-06-13T16:23:06.577171Z info klog "msg"="-----------------------------------------------------------------"
2022-06-13T16:23:06.610615Z info klog tls-provider/root-ca-watcher "msg"="loading root CAs bundle" "file"="/var/run/secrets/istio-csr/ca.crt" 2022-06-13T16:23:06.610651Z info klog manager "msg"="Starting server" "addr"={"IP":"::","Port":9402,"Zone":""} "kind"="metrics" "path"="/metrics" 2022-06-13T16:23:06.610664Z info klog tls-provider/root-ca-watcher "msg"="updating root CAs from file" "file"="/var/run/secrets/istio-csr/ca.crt" 2022-06-13T16:23:06.610760Z info klog manager "msg"="Starting server" "addr"={"IP":"::","Port":6060,"Zone":""} "kind"="health probe" 2022-06-13T16:23:06.610813Z info klog attempting to acquire leader lease istio-system/istio-csr... 2022-06-13T16:23:06.639640Z info klog successfully acquired lease istio-system/istio-csr 2022-06-13T16:23:06.640303Z info klog manager/events "msg"="Normal" "message"="cert-manager-istio-csr-7bf86d579f-5xg8f_b59ef618-0e58-4001-9b52-b8489467929d became leader" "object"={"kind":"ConfigMap","namespace":"is tio-system","name":"istio-csr","uid":"3f358c3a-79ea-47b4-a4a2-96b886847ad6","apiVersion":"v1","resourceVersion":"5297621"} "reason"="LeaderElection" 2022-06-13T16:23:06.640366Z info klog manager/events "msg"="Normal" "message"="cert-manager-istio-csr-7bf86d579f-5xg8f_b59ef618-0e58-4001-9b52-b8489467929d became leader" "object"={"kind":"Lease","namespace":"istio- system","name":"istio-csr","uid":"aa2672a4-54b5-4dfb-856a-4e58ae2e8b76","apiVersion":"coordination.k8s.io/v1","resourceVersion":"5297622"} "reason"="LeaderElection" 2022-06-13T16:23:06.640380Z info klog manager/controller/configmap "msg"="Starting EventSource" "reconciler group"="" "reconciler kind"="ConfigMap" "source"="kind source: v1.PartialObjectMetadata" 2022-06-13T16:23:06.640411Z info klog manager/controller/configmap "msg"="Starting EventSource" "reconciler group"="" "reconciler kind"="ConfigMap" "source"="kind source: v1.Namespace" 2022-06-13T16:23:06.640456Z info klog manager/controller/configmap "msg"="Starting EventSource" "reconciler group"="" "reconciler kind"="ConfigMap" "source"="channel source: 0xc000eb0b40" 2022-06-13T16:23:06.640484Z info klog manager/controller/configmap "msg"="Starting Controller" "reconciler group"="" "reconciler kind"="ConfigMap" 2022-06-13T16:23:06.741832Z info klog manager/controller/configmap "msg"="Starting workers" "reconciler group"="" "reconciler kind"="ConfigMap" "worker count"=1 2022-06-13T16:23:06.843105Z info klog controller/configmap "msg"="updating ConfigMap data" "configmap"="istio-ca-root-cert" "namespace"="platform-delivery" 2022-06-13T16:23:06.857005Z info klog controller/configmap "msg"="updating ConfigMap data" "configmap"="istio-ca-root-cert" "namespace"="istio-addons-ingress" 2022-06-13T16:23:06.870137Z info klog controller/configmap "msg"="updating ConfigMap data" "configmap"="istio-ca-root-cert" "namespace"="cert-manager" 2022-06-13T16:23:06.884143Z info klog controller/configmap "msg"="creating configmap with root CA data" "configmap"="istio-ca-root-cert" "namespace"="kube-public" 2022-06-13T16:23:06.897590Z info klog controller/configmap "msg"="updating ConfigMap data" "configmap"="istio-ca-root-cert" "namespace"="calico-system" 2022-06-13T16:23:06.909220Z info klog controller/configmap updating ConfigMap data"istio-ca-root-cert" "namespace"="gatekeeper-system" 2022-06-13T16:23:06.918472Z info klog controller/configmap "msg"="updating ConfigMap data" "configmap"="istio-ca-root-cert" "namespace"="istio-ingress" 2022-06-13T16:23:06.928778Z info klog controller/configmap "msg"="updating ConfigMap data" "configmap"="istio-ca-root-cert" "namespace"="istio-system"

JoshVanL commented 2 years ago

Hi @ceastman-r7, do you mean it is repeated once for each namespace, or is continuously logged for every namespace over and over again?

If the latter, then this suggests that some other entity (like istiod) is also attempting to write to these ConfigMaps which will thrash both controllers.

ceastman-r7 commented 2 years ago

continuously logging.

Oh could istiod be trying to reset it back to the self signed certificate?

JoshVanL commented 2 years ago

Yes, sounds like to me istiod might be missing this configuration

https://github.com/cert-manager/istio-csr/blob/4200304ed29471f4bde2c499da7e60614e69efeb/hack/istio-config-1.13.4.yaml#L20

ceastman-r7 commented 2 years ago

yeah i havent switched the istio config yet. thank you.

ceastman-r7 commented 2 years ago

Do you have a values.yaml override file that can be used with helm instead of an istiooperating yaml to accomplish: https://github.com/cert-manager/istio-csr/blob/main/docs/istio-config-getting-started.yaml

JoshVanL commented 2 years ago

This thread should be of help 🙂

https://github.com/cert-manager/istio-csr/issues/113