spiffe / spire-controller-manager

Kubernetes controller manager that reconciles workload registration and federation relationships.
Apache License 2.0
54 stars 37 forks source link

Clusterstaticentries don't get created #441

Open MattiasGees opened 20 hours ago

MattiasGees commented 20 hours ago

I am having some problems with clusterstaticentries not being created. I have SPIRE deployed with the helm-charts. I am using the following version:

spire-server: 1.11.0 spire-controller-manager: 0.6.0 spire-crd (helm): 0.5.0 spire (helm): 0.24.0

I am using the following helm values:

global:
  spire:
    clusterName: playground
    trustDomain: spire.internal.mattiasgees.be
    jwtIssuer: https://oidc-discovery.${domain}
    recommendations:
      enabled: false
    namespaces:
      create: false
    strictMode: true
    telemetry:
      prometheus:
        enabled: true
        podMonitor:
          enabled: true

spire-server:
  logLevel: debug
  nodeAttestor:
    joinToken:
      enabled: true
  unsupportedBuiltInPlugins:
    nodeAttestor:
      x509pop:
        plugin_data:
          ca_bundle_path: "/run/spire/ca-bundle/ca.crt"
  extraVolumes:
  - name: ca-bundle
    secret:
      secretName: laptop-tls
  extraVolumeMounts:
  - name: ca-bundle
    mountPath: /run/spire/ca-bundle
    readOnly: true
  caSubject:
    country: UK
    organization: My Org
    commonName: spire.internal.mattiasgees.be
  controllerManager:
    enabled: true
    identities:
      clusterSPIFFEIDs:
        default:
          autoPopulateDNSNames: true
          workloadSelectorTemplates:
          - "k8s:pod-name:{{ .PodMeta.Name }}"
          - "k8s:ns:{{ .PodMeta.Namespace }}"
          - "k8s:node-name:{{ .NodeMeta.Name }}"
  service:
    type: LoadBalancer

I am joining an external host through the x509pop and that works without any problem. I can see it showing up in the SPIRE server as spiffe://spire.internal.mattiasgees.be/spire/agent/x509pop/<fingerprint>

The Kubernetes auto-discovery with ClusterSPIFFEIds works correctly, but when I add a clusterStaticEntry for a workload running on that x509pop, it doesn't get created in the SPIRE server. The clusterStaticEntry looks the following:

apiVersion: spire.spiffe.io/v1alpha1
kind: ClusterStaticEntry
metadata:
  name: company-laptop-workload
spec:
  spiffeID: "spiffe://spire.internal.mattiasgees.be/static/macbook/company/workload"
  parentID: "spiffe://spire.internal.mattiasgees.be/spire/agent/x509pop/<fingerprint>"
  selectors:
    - unix:uid:505

In the SPIRE controller manager logs, I can see that the reconciliation is starting, but nothing else. It doesn't show any errors or further output.

2024-11-01T07:40:11Z    DEBUG   Triggering reconciliation   {"controller": "clusterstaticentry", "controllerGroup": "spire.spiffe.io", "controllerKind": "ClusterStaticEntry", "ClusterStaticEntry": {"name":"company-laptop-workload"}, "namespace": "", "name": "company-laptop-workload", "reconcileID": "423258f0-13fe-49c8-afb5-274807f2c846"}
azdagron commented 12 hours ago

Can you provide the configuration that is ultimately passed into the controller?

kfox1111 commented 6 hours ago

The chart by default does not watch all entries, but instead only watches ones with a generated className.

There's 4 ways to go from here.

  1. If you create the static entries through the chart values, it will "just work" (ex: https://github.com/spiffe/helm-charts-hardened/blob/main/charts/spire/charts/spire-server/values.yaml#L670)
  2. You can add the className setting to the static entry. can see one of the loaded clusterspiffeid's for the value.
  3. You can do 2, along with specify the exact className you want the chart to use: https://github.com/spiffe/helm-charts-hardened/blob/main/charts/spire/charts/spire-server/values.yaml#L510-L511
  4. You can have your instance honor unclassed objects too: https://github.com/spiffe/helm-charts-hardened/blob/main/charts/spire/charts/spire-server/values.yaml#L512-L513