pulumi / pulumi-kubernetes

A Pulumi resource provider for Kubernetes to manage API resources and workloads in running clusters
https://www.pulumi.com/docs/reference/clouds/kubernetes/
Apache License 2.0
414 stars 116 forks source link

Using a k8s CRD fails due to ArgoCD App k8s service still not ready even though the ArgoCD App is marked as healthy/synced and the helm chart has parameter wait #3316

Open cooervo opened 1 day ago

cooervo commented 1 day ago

What happened?

Getting error when running pulumi update:

 Preview failed: resource "urn:pulumi:mc-predev::foo-infra::kubernetes:external-secrets.io/v1alpha1:ClusterSecretStore::
dev-cluster-secret-store" was not successfully created by the Kubernetes API server: conversion webhook for external-secrets.io/v1alpha1,
 Kind=ClusterSecretStore failed: Post "https://external-secrets-webhook.external-secrets.svc:443/convert?timeout=30s": 
service "external-secrets-webhook" not found

Example

I'm getting the above error when trying to install in same pulumi update the following 2 resources:

// The ArgoCD app installing the helm chart for external-secrets
 const externalSecretsApp = new Application.argoproj.v1alpha1.Application(
    `${env}-external-secrets-app`,
    {
      name: 'external-secrets',
      namespace: 'external-secrets',
      sources: [
        {
          path: 'kubernetes/helm/charts/external-secrets/',
          helm: {
            parameters: [
              {
                name: 'wait',
                value: 'true',
              },
            ],
            valueFiles: [`../../values/external-secrets/${env}.values.yaml`],
            ...
          repoURL: INFRA_REPO_URL,
          targetRevision: env,
        },
      ],
    ...
    },
    {
      provider,
      dependsOn: [...],
    },
  );

// The CRD I previously got from using crd2pulumi --nodejsPath ClusterSecretStore ClusterSecretStore.yaml    
  const clusterSecretStore =
    new ClusterSecretStore.external_secrets.v1alpha1.ClusterSecretStore(
      `${env}-cluster-secret-store`,
      {
        metadata: {
          name: 'gcp-cluster-secret-store',
        },
        spec: {
          provider: {
            // gcpsm = GCP  Secret Manager
            gcpsm: {
              projectID: GCP_PROJECT,
            },
          },
        },
      },
      { dependsOn: [...], provider },
    );

Even though first resource, externalSecretsApp, is dependency in dependsOn of second resource, clusterSecretStore, I still get the error:

 Preview failed: resource "urn:pulumi:mc-predev::foo-infra::kubernetes:external-secrets.io/v1alpha1:ClusterSecretStore::
dev-cluster-secret-store" was not successfully created by the Kubernetes API server: conversion webhook for external-secrets.io/v1alpha1, 
Kind=ClusterSecretStore failed: Post "https://external-secrets-webhook.external-secrets.svc:443/convert?timeout=30s": 
service "external-secrets-webhook" not found

Output of pulumi about

pulumi about
CLI          
Version      3.139.0
Go Version   go1.23.3
Go Compiler  gc

Plugins
KIND      NAME        VERSION
resource  command     1.0.1
resource  gcp         8.0.0
resource  kubernetes  4.18.1
language  nodejs      unknown
resource  random      4.16.7
resource  std         1.7.3

Host     
OS       darwin
Version  14.6.1
Arch     arm64

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

cooervo commented 1 day ago

I also tried wrapping the CRD in a custom resource but still get same error:

const clusterSecretStore = new k8s.apiextensions.CustomResource(
    `${env}-cluster-secret-store`,
    {
      apiVersion: 'external-secrets.io/v1alpha1',
      kind: 'ClusterSecretStore',
      metadata: {
        name: 'gcp-cluster-secret-store',
      },
      spec: {
        provider: {
          // gcpsm = GCP  Secret Manager
          gcpsm: {
            projectID: GCP_PROJECT,
          },
        },
      },
    },
    { dependsOn: [externalSecretsApp], provider },
  );