Same client certificate in different `Secret`s used in different services causes invalid configuration

[randmonkey]

Is there an existing issue for this?

• [X] I have searched the existing issues

Current Behavior

Background: customer wanted to use the same certificate for services in different namespaces as mTLS client certificate.

If we create multiple secrets with the same certificate-key pair, and use the secrets as client certificates via konghq.com/client-certificate, KIC will fail to apply the configuration and the error message will be empty so we cannot parse it.

declarative config is invalid: {}

This is because we will merge the certificates with the same content and only reserve one Kong certificate, but the client_certificate ID used in services are from the referred secret UID. So some of the services will refer to a non-exist certificate, which makes the Kong configuration invalid.

Expected Behavior

Both services can refer to the correct certificate in this scenario, or other ways to use the same certificates for services in different namespaces.

Steps To Reproduce

(1) Create 2 tls secrets with the same key-cert pair:

kubectl create secret tls client-cert-1 --cert="a.crt" --key="a.key"
kubectl create secret tls client-cert-2 --cert="a.crt" --key="a.key"

(2) Create 2 services using the 2 secrets as client certificate, and 2 ingresses using them as backends:

echo 'apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpbin-deployment-1
  labels:
    app: httpbin
spec:
  replicas: 1
  selector:
    matchLabels:
      app: httpbin
  template:
    metadata:
      labels:
        app: httpbin
    spec:
      containers:
      - name: httpbin
        image: kong/httpbin:0.1.0
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: httpbin
  name: httpbin-deployment-1
  annotations:
    konghq.com/client-cert: "client-cert-1"
    konghq.com/protocol: "https"
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: httpbin
  type: ClusterIP
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: httpbin-ingress-1
  annotations:
    konghq.com/strip-path: "true"
spec:
  ingressClassName: kong
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: httpbin-deployment
            port:
              number: 80' | kubectl apply -f -

echo 'apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpbin-deployment-2
  labels:
    app: httpbin
spec:
  replicas: 1
  selector:
    matchLabels:
      app: httpbin
  template:
    metadata:
      labels:
        app: httpbin
    spec:
      containers:
      - name: httpbin
        image: kong/httpbin:0.1.0
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: httpbin
  name: httpbin-deployment-2
  annotations:
    konghq.com/client-cert: "client-cert-2"
    konghq.com/protocol: "https"
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: httpbin
  type: ClusterIP
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: httpbin-ingress-2
  annotations:
    konghq.com/strip-path: "true"
spec:
  ingressClassName: kong
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: httpbin-deployment
            port:
              number: 80' | kubectl apply -f -

Kong Ingress Controller version

2.12.3, but still exist in 3.2

Kubernetes version

Not relevant

Anything else?

No response

[lahabana]

Can we have an example manifest of this please?

[randmonkey]

Can we have an example manifest of this please?

Added example.