search

jaconi-io / secret-file-provider

K8s workload sidecar to copy K8s secret content to filesystem
MIT License
1 stars 0 forks source link

Unable to delete secrets due to finalizers #85

Open jfiedler-jaconi opened 9 months ago

jfiedler-jaconi commented 9 months ago

The secret-file-provider seems to add finalizers to secrets that it collected at one point. We ended up with lots of finalizers at the secret level (see below). These finalizers prevented us from deleting the secrets.

apiVersion: v1
data:
  CLIENT_ID: redacted
  CLIENT_SECRET: redacted
kind: Secret
metadata:
  creationTimestamp: "2023-03-24T08:17:03Z"
  deletionGracePeriodSeconds: 0
  deletionTimestamp: "2024-01-22T13:20:25Z"
  finalizers:
  - jaconi.io/secret-file-provider-sor-for-features-796c5f5bc-jmtg4
  - jaconi.io/secret-file-provider-or-for-features-78ff767cd5-r8glv
  - jaconi.io/secret-file-provider-or-for-features-67557bfd9f-b9k7j
  - jaconi.io/secret-file-provider-event-processor-5d6cb9d794-cc9cp
  - jaconi.io/secret-file-provider-or-for-features-6789c66795-xbtmr
  - jaconi.io/secret-file-provider-or-for-features-77d86dc5ff-rbn9t
  - jaconi.io/secret-file-provider-or-for-features-77d86dc5ff-dbwqk
  - jaconi.io/secret-file-provider-or-for-features-77d86dc5ff-7k7vj
  - jaconi.io/secret-file-provider-or-for-features-77d86dc5ff-xh2js
  - jaconi.io/secret-file-provider-or-for-features-77d86dc5ff-8rf2d
  - jaconi.io/secret-file-provider-or-for-features-77d86dc5ff-vhs9l
  - jaconi.io/secret-file-provider-or-for-features-77d86dc5ff-8ghpw
  - jaconi.io/secret-file-provider-event-processor-5fbff698f9-lwm6b
  - jaconi.io/secret-file-provider-event-processor-5fbff698f9-7p67j
  - jaconi.io/secret-file-provider-or-for-features-7d848fd765-22d76
  - jaconi.io/secret-file-provider--event-processor-cccf8dc94-q9cgp
  - jaconi.io/secret-file-provider-or-for-features-68b8d55dc6-p5km2
  - jaconi.io/secret-file-provider-event-processor-675f6d964c-bt4jk
  - jaconi.io/secret-file-provider-event-processor-6489585f55-hctll
  - jaconi.io/secret-file-provider-event-processor-5d478ff796-k226q
  - jaconi.io/secret-file-provider-event-processor-6d56bf4cd5-gqcs2
  - jaconi.io/secret-file-provider-event-processor-5d5b76b85c-z6n6z
  - jaconi.io/secret-file-provider-sor-for-features-bb7cb8ff9-qnx8s
  - jaconi.io/secret-file-provider-or-for-features-545fc8666f-9hffr
  - jaconi.io/secret-file-provider-event-processor-854b95cfd8-cr499
  - jaconi.io/secret-file-provider-or-for-features-5cdf9c44f8-6l89g
  - jaconi.io/secret-file-provider-g-event-processor-f78576cc-5t47f
  labels:
    app: keycloak
  name: keycloak-client-secret-thing-event-processor-bn
  namespace: kiwios
  ownerReferences:
  - apiVersion: keycloak.org/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: KeycloakClient
    name: thing-event-processor-bn
    uid: a682b685-582c-4c7d-876d-dd1e443f9f19
  resourceVersion: "1563742131"
  uid: 0aa80df2-5baf-4d27-8d3b-7ea92fb6ace5
type: Opaque
mhartmann-jaconi commented 9 months ago

This is a conceptual problem, as we do not require a shutdown hook (which would take care for the cleanup if the pod is dropped). However, this is also not a perfect solution, as the pod might get killed without graceful termination. As a quickfix, I would propose to provide a new version of this sidecar, where the secret-deletion handling is optional. For the long-term solution we might need some more thoughts...

mhartmann-jaconi commented 9 months ago

Proposal: We remove the finalizer handling completely and therefore get rid of the problem at all.

Instead we can use a sync job (running e.g. every 10 minutes) to sync the cluster state with the filesystem-state, to make sure we did not loose any deletion event.