Hostname mismatch on ipv6 kubernetes clusters

[Gitopolis]

extension-init comes as a post-install hook in the linkerd-smi helm chart. This configuration works in ipv4 cluster, but fails in ipv6

Job manifest:

apiVersion: batch/v1
kind: Job
metadata:
  annotations:
    helm.sh/hook: post-install
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
    helm.sh/hook-weight: '1'
  generation: 1
  labels:
    app.kubernetes.io/name: namespace-metadata
    app.kubernetes.io/part-of: linkerd-smi
    app.kubernetes.io/version: v0.2.7
  name: namespace-metadata
  namespace: linkerd
  resourceVersion: '82629'
  uid: ac2bfd1c-727f-46f0-905b-88236b59ecff
spec:
  backoffLimit: 6
  completionMode: NonIndexed
  completions: 1
  manualSelector: false
  parallelism: 1
  podReplacementPolicy: TerminatingOrFailed
  selector:
    matchLabels:
      batch.kubernetes.io/controller-uid: ac2bfd1c-727f-46f0-905b-88236b59ecff
  suspend: false
  template:
    metadata:
      creationTimestamp: null
      labels:
        app.kubernetes.io/name: namespace-metadata
        app.kubernetes.io/part-of: linkerd-smi
        app.kubernetes.io/version: v0.2.7
        batch.kubernetes.io/controller-uid: ac2bfd1c-727f-46f0-905b-88236b59ecff
        batch.kubernetes.io/job-name: namespace-metadata
        controller-uid: ac2bfd1c-727f-46f0-905b-88236b59ecff
        job-name: namespace-metadata
    spec:
      containers:
        - args:
            - '--extension'
            - smi
            - '--namespace'
            - linkerd
            - '--linkerd-namespace'
            - linkerd
          image: cr.l5d.io/linkerd/extension-init:v0.1.1
          imagePullPolicy: IfNotPresent
          name: namespace-metadata
          resources: {}
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop:
                - ALL
            readOnlyRootFilesystem: true
            runAsNonRoot: true
            runAsUser: 65534
            seccompProfile:
              type: RuntimeDefault
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Never
      schedulerName: default-scheduler
      securityContext:
        seccompProfile:
          type: RuntimeDefault
      serviceAccount: namespace-metadata
      serviceAccountName: namespace-metadata
      terminationGracePeriodSeconds: 30

2024-10-11T03:01:15.417751Z INFO linkerd_extension_init: patching namespace linkerd
2024-10-11T03:01:15.424705Z ERROR kube_client::client::builder: failed with error error trying to connect: error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:2092:: hostname mismatch
Error: HyperError: error trying to connect: error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:2092:: hostname mismatch
Caused by:
0: error trying to connect: error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:2092:: hostname mismatch
1: error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:2092:: hostname mismatch
2: error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:2092:
3: error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:2092:

Steps to reproduce:

• Create EKS cluster on AWS with IPv6 cluster IP address family. Kubernetes version 1.31

• Install linkerd and linkerd-smi extension. My installation includes cert-manager, but I don't think this is related. Anyway: kustomization yaml:

  apiVersion: kustomize.config.k8s.io/v1beta1
  kind: Kustomization
  helmCharts:
  - name: linkerd-crds
  repo: https://helm.linkerd.io/edge
  version: 2024.10.2
  valuesInline:
    enableHttpRoutes: false
  - name: linkerd-smi
  repo: https://linkerd.github.io/linkerd-smi
  version: 1.0.4
  namespace: linkerd
  valuesInline:
    namespaceMetadata.image.tag: v0.1.1
  - name: linkerd-control-plane
  repo: https://helm.linkerd.io/edge
  version: 2024.10.2
  namespace: linkerd
  valuesFile: values.yaml
  
  resources:
  - ns.yaml
  - ca.yaml
  - certs.yaml

  values.yaml

  identity:
  externalCA: true
  issuer:
    scheme: kubernetes.io/tls
  proxyInjector:
  externalSecret: true
  injectCaFrom: linkerd/linkerd-proxy-injector
  profileValidator:
  externalSecret: true
  injectCaFrom: linkerd/linkerd-sp-validator
  policyValidator:
  externalSecret: true
  injectCaFrom: linkerd/linkerd-policy-validator
  disableIPv6: false

  ns.yaml

  apiVersion: v1
  kind: Namespace
  metadata:
  name: linkerd
  annotations:
    linkerd.io/inject: disabled
    linkerd.io/is-control-plane: "true"
    linkerd.io/control-plane-ns: linkerd
    config.linkerd.io/admission-webhooks: disabled
    pod-security.kubernetes.io/enforce: privileged

  certs.yaml

  apiVersion: cert-manager.io/v1
  kind: Issuer
  metadata:
  name: webhook-issuer
  namespace: linkerd
  spec:
  ca:
    secretName: webhook-issuer-tls
  ---
  apiVersion: cert-manager.io/v1
  kind: Certificate
  metadata:
  name: linkerd-policy-validator
  namespace: linkerd
  spec:
  secretName: linkerd-policy-validator-k8s-tls
  duration: 24h0m0s
  renewBefore: 1h0m0s
  revisionHistoryLimit: 3
  issuerRef:
    name: webhook-issuer
    kind: Issuer
  commonName: linkerd-policy-validator.linkerd.svc
  dnsNames:
  - linkerd-policy-validator.linkerd.svc
  privateKey:
    algorithm: ECDSA
    encoding: PKCS8
  usages:
  - server auth
  ---
  apiVersion: cert-manager.io/v1
  kind: Certificate
  metadata:
  name: linkerd-proxy-injector
  namespace: linkerd
  spec:
  secretName: linkerd-proxy-injector-k8s-tls
  duration: 24h0m0s
  renewBefore: 1h0m0s
  revisionHistoryLimit: 3
  issuerRef:
    name: webhook-issuer
    kind: Issuer
  commonName: linkerd-proxy-injector.linkerd.svc
  dnsNames:
  - linkerd-proxy-injector.linkerd.svc
  privateKey:
    algorithm: ECDSA
  usages:
  - server auth
  ---
  apiVersion: cert-manager.io/v1
  kind: Certificate
  metadata:
  name: linkerd-sp-validator
  namespace: linkerd
  spec:
  secretName: linkerd-sp-validator-k8s-tls
  duration: 24h0m0s
  renewBefore: 1h0m0s
  revisionHistoryLimit: 3
  issuerRef:
    name: webhook-issuer
    kind: Issuer
  commonName: linkerd-sp-validator.linkerd.svc
  dnsNames:
  - linkerd-sp-validator.linkerd.svc
  privateKey:
    algorithm: ECDSA
  usages:
  - server auth

  ca.yaml

  apiVersion: cert-manager.io/v1
  kind: Certificate
  metadata:
  name: linkerd-identity-issuer
  namespace: linkerd
  spec:
  isCA: true
  commonName: identity.linkerd.cluster.local
  secretName: linkerd-identity-issuer
  duration: 48h0m0s
  revisionHistoryLimit: 3
  privateKey:
    algorithm: ECDSA
    size: 256
  issuerRef:
    name: root-linkerd-issuer
    kind: ClusterIssuer
    group: cert-manager.io
  dnsNames:
  - identity.linkerd.cluster.local
  usages:
  - cert sign
  - crl sign
  - server auth
  - client auth
  ---
  apiVersion: cert-manager.io/v1
  kind: Certificate
  metadata:
  name: linkerd-webhook-issuer
  namespace: linkerd
  spec:
  isCA: true
  commonName: webhook.linkerd.cluster.local
  secretName: webhook-issuer-tls
  duration: 48h0m0s
  revisionHistoryLimit: 3
  privateKey:
    algorithm: ECDSA
    size: 256
  issuerRef:
    name: root-linkerd-issuer
    kind: ClusterIssuer
    group: cert-manager.io
  dnsNames:
  - webhook.linkerd.cluster.local
  usages:
  - cert sign
  - crl sign
  - server auth
  - client auth
  ---
  apiVersion: trust.cert-manager.io/v1alpha1
  kind: Bundle
  metadata:
    name: linkerd-identity-trust-roots
  spec:
  sources:
  - secret:
      name: "linkerd-trust-anchor"
      key: "ca.crt"
  target:
    configMap:
      key: "ca-bundle.crt"

[alpeb]

Thanks for the detailed report; I was able to repro the issue :+1: We'll fix this, but in the meantime you can run helm with the --no-hooks flag to avoid triggering that failing Job. That job only adds some metadata into the linkerd-smi after it's created by helm. So to replace that, you can have your pipeline create the linkerd-smi namespace with the appropriate metadata:

apiVersion: v1
kind: Namespace
metadata:
  labels:
    kubernetes.io/metadata.name: linkerd-smi
    linkerd.io/extension: smi
    name: linkerd-smi
    pod-security.kubernetes.io/enforce: privileged
  name: linkerd-smi