Azure / AKS

Azure Kubernetes Service
1.92k stars 284 forks source link

[BUG]After the SSL custom CA configuration on Red Hat OpenShift cluster, all the Pods belong to "Microsoft Defender for Container" got error: x509: certificate signed by unknown authority #4236

Closed Harry1le closed 3 weeks ago

Harry1le commented 3 weeks ago

****Describe the bug**** After the SSL custom CA configuration on Red Hat OpenShift cluster, all the Pods belong to "Microsoft Defender for Container" got error: x509: certificate signed by unknown authority'

The pods in "mdc" namespace: microsoft-defender-publisher-ds-*** is failling due to 'panic: Error encountered during client initialization Post 'https://..........oms.opinsights.azure.com/AgentService.svc/LinuxAgentTopologyRequest': x509: certificate signed by unknown authority '

****To Reproduce**** I have followed the document below but stuck in Certificate injection using Operators. https://docs.openshift.com/container-platform/4.6/networking/configuring-a-custom-pki.html

The defender for container configuration follow: https://learn.microsoft.com/en-us/azure/defender-for-cloud/defender-for-containers-enable?tabs=k8s-deploy-cli%2Ck8s-deploy-asc%2Ck8s-verify-asc%2Ck8s-remove-cli%2Caks-removeprofile-api&pivots=defender-for-container-arc Kubernetes distributions (using Azure Arc-enabled Kubernetes)


I have tried to create the custom CA (cert) configmap and update that to the daemonset of microsoft-defender-publisher-ds-. But the error still persists.

1.Create a ConfigMap with Your Custom CA Certificate. -Save your CA certificate to a file, for example, ca.crt.

-Use the following command to create a ConfigMap containing this certificate file, ensuring it is deployed in the correct namespace (often kube-system, but this depends on where your DaemonSet is located): oc create configmap custom-ca --from-file=ca.crt=./path/to/ca.crt -n kube-system

2.Update the DaemonSet to Use the Custom CA Certificate Modify the microsoft-defender-publisher-ds DaemonSet so that its Pods can load and utilize the CA certificate: -Edit the DaemonSet configuration: oc edit daemonset microsoft-defender-publisher-ds -n kube-system

-Add the ConfigMap as a Volume within the DaemonSet’s Pod template: volumes:

-Ensure that the Volume is mounted in each container at the appropriate directory (such as /etc/pki/ca-trust/source/anchors/, a common directory for CA certificates on Red Hat systems): volumeMounts:

-Configure the container to update the CA certificate store on startup, which can be achieved by adding a startup command: lifecycle: postStart: exec: command: ["/bin/bash", "-c", "update-ca-trust extract"]

****Expected behavior** I want to make some change on the network (for Netskope to work) external to OpenShift. The SSL custom CA was needed after the change. I think there should be some configuration on for the daemonset/configmap for the "Defender for Container" pods to use this new cert.

Environment (please complete the following information):

Harry1le commented 3 weeks ago

I closed this issue because I should not the one to open this ticket.