pulumi / pulumi-kubernetes

A Pulumi resource provider for Kubernetes to manage API resources and workloads in running clusters
https://www.pulumi.com/docs/reference/clouds/kubernetes/
Apache License 2.0
404 stars 115 forks source link

failed to determine if the following GVK is namespaced if CRD is created in the same run #3176

Open ecerulm opened 3 weeks ago

ecerulm commented 3 weeks ago

What happened?

I'm deploying cert-manager helm release + a ClusterIssuer resource

cert-manager helm release creates the ClusterIssuer CRD, but it seems that the pulumi.ConfigGroup that creates the ClusterIssuer resources tries to check some properties of the ClusterIssuer CRD before it's actually created although there is depends_on to the cert-manager.

    Exception: marshaling properties: awaiting input property "resources": failed to determine if the following GVK is namespaced: cert-manager.io/v1, Kind=ClusterIssuer

If I rerun pulumi up -y --skip-preview after then the ClusterIssuer will be created fine. That's why I think it's a timing issue between the creation of the ClusterIssuer CRD and the actual ClusterIssuer resource.

Example

cert_manager = helmv3.Release(
    "cert_manager",
    helmv3.ReleaseArgs(
        # https://cert-manager.io/docs/installation/helm/
        # https://artifacthub.io/packages/helm/cert-manager/cert-manager
        # https://github.com/cert-manager/cert-manager
        name="cert-manager",
        chart="cert-manager",
        namespace=namespace.id,
        version="1.15.2",
        repository_opts=helmv3.RepositoryOptsArgs(
            repo="https://charts.jetstack.io",
        ),
        values=cert_manager_helm_values,
    ),
    opts=pulumi.ResourceOptions(
        provider=kubernetes_provider,
    ),
)

def generate_clusterissuer_manifest(name, server):
    def func(args):
        template = env.get_template("letsencrypt-clusterissuer.j2.yaml")
        rendered = template.render(
            domain_name=args["domain_name"],
            region=args["region"],
            zone_id=args["zone_id"],
            name=name,
            server=server,
        )
        return rendered

    return pulumi.Output.all(
        domain_name=domain_name,
        region=region,
        zone_id=zone.id,
    ).apply(func)

# https://www.pulumi.com/registry/packages/kubernetes/api-docs/yaml/configgroup/
letsencrypt_stagin_cluster_issuer_cg = kubernetes.yaml.v2.ConfigGroup(
    "letsencrypt-staging",
    yaml=generate_clusterissuer_manifest(
        name="letsencrypt-staging",
        server="https://acme-staging-v02.api.letsencrypt.org/directory",
    ),
    opts=pulumi.ResourceOptions(
        depends_on=[
            cert_manager,
        ],
        provider=kubernetes_provider,
    ),
)

Output of pulumi about

pulumi about
CLI          
Version      3.129.0
Go Version   go1.22.6
Go Compiler  gc

Plugins
KIND      NAME        VERSION
resource  aws         6.49.1
resource  eks         2.7.8
resource  kubernetes  4.17.1
language  python      unknown
resource  random      4.16.3

Host     
OS       darwin
Version  14.6.1
Arch     x86_64

This project is written in python: executable='/Users/xxx/git/pulumi-aws-ecerulm/venv/bin/python' version='3.12.5'

...

Backend        
Name           xxxxx
URL            file://~
User           xxxx
Organizations  
Token type     personal

Dependencies:
NAME           VERSION
Jinja2         3.1.4
pip            24.2
pulumi_eks     2.7.8
pulumi_random  4.16.3
setuptools     72.2.0
wheel          0.44.0

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

ecerulm commented 3 weeks ago

The workaround I use now is to create the CRDs myself with another kubernetes.yaml.v2.ConfigFile with just the CRDs

# https://www.pulumi.com/registry/packages/kubernetes/api-docs/yaml/configfile/
crds = kubernetes.yaml.v2.ConfigFile(
    "letsencrypt-prod",
    file="files/cert-manager.crds.yaml",
    opts=pulumi.ResourceOptions(
        depends_on=[

        ],
        provider=kubernetes_provider,
    ),
) 

then I set installCRDs: false for the cert-manager helm values, and make the ClusterIssuer depend on the crds ConfigFile and cert-manager's Release