crossplane-contrib / function-go-templating

A Go templating composition function
https://crossplane.io
Apache License 2.0
57 stars 37 forks source link

Composition doesn't render/generate all resources when it is made up of multiple complex/nested compositions #124

Closed Checksumz closed 1 month ago

Checksumz commented 1 month ago

What happened?

When I try to use 2 complex/nested compositions with go-templating they do not render together.

If i comment one of them out the other successfully gets generated. In the below case when i don't comment any of the templates out only the step declared last is rendered (generated). When I converted my Project resource(nested composition) from go-templating to patch and transform function, it started rendering(generating managed resources)

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: nw.gcp.odysseycloud.io
  labels:
    meta.odysseycloud.io/provider: GCP
    meta.odysseycloud.io/service: Nw
spec:
  compositeTypeRef:
    apiVersion: gcp.odysseycloud.io/v1alpha1
    kind: Nw
  mode: Pipeline
  pipeline:
  - step: gke-cluster-setup
    functionRef:
      name: function-go-templating
    input:
      apiVersion: gotemplating.fn.crossplane.io/v1beta1
      kind: GoTemplate
      source: Inline
      inline:
        template: |
          apiVersion: gcp.odysseycloud.io/v1alpha2
          kind: Project
          metadata:
            name: {{ .observed.composite.resource.metadata.name }}
            annotations:
              gotemplating.fn.crossplane.io/composition-resource-name: {{ .observed.composite.resource.metadata.name }}
          spec:
            name: {{ .observed.composite.resource.metadata.name }}
            env: {{ .observed.composite.resource.spec.env }}
            team: {{ .observed.composite.resource.spec.team }}
            services: 
              {{- range $serviceType, $enabled := .observed.composite.resource.spec.services }}
              {{ $serviceType }}: {{ $enabled }}
              {{- end }}
            compositionSelector:
              matchLabels:
                meta.odysseycloud.io/provider: GCP
                meta.odysseycloud.io/service: Project
  - step: gke-nw-setup
    functionRef:
      name: function-go-templating
    input:
      apiVersion: gotemplating.fn.crossplane.io/v1beta1
      kind: GoTemplate
      source: Inline
      inline:
        template: |
          apiVersion: gcp.odysseycloud.io/v1alpha1
          kind: Network
          metadata:
            name: {{ .observed.composite.resource.metadata.name }}
            annotations:
              gotemplating.fn.crossplane.io/composition-resource-name: {{ .observed.composite.resource.metadata.name }}
          spec:
            env: {{ .observed.composite.resource.spec.env }}
            team: {{ .observed.composite.resource.spec.team }}
            locations:
            {{- range .observed.composite.resource.spec.locations -}}
              - {{- . -}}
            {{- end -}}
            dns:
              subDomain: {{ .observed.composite.resource.spec.dns.subDomain }}
              visibility: {{ .observed.composite.resource.spec.dns.visibility }}
            compositionSelector:
              matchLabels:
                meta.odysseycloud.io/provider: GCP
                meta.odysseycloud.io/service: Network
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: nws.gcp.odysseycloud.io
spec:
  group: gcp.odysseycloud.io
  names:
    kind: Nw
    plural: nws
  claimNames:
    kind: NwClaim
    plural: nwclaims
  versions:
  - name: v1alpha1
    served: true
    referenceable: true
    schema:
      openAPIV3Schema: 
        type: object
        properties:
          spec:
            type: object
            properties:
              env:
                type: string
                description: "Supported environments of GCP project: dev, stg, prod"
              team:
                type: string
                description: GCP project team
              services:
                type: object
                description: Map of services for project
                additionalProperties: 
                  type: boolean
              locations:
                type: array
                description: List of locations for subnet
                items:
                  type: string
              dns:
                type: object
                description: DNS details of project
                properties:
                  subDomain:
                    description: The subDomain of GCP DNS.
                    type: string
                  visibility:
                    description: The visibility of GCP DNS.
                    type: string
                    default: private
                required:
                - visibility
            required:
            - env
            - team
            - services
            - locations
            - dns
apiVersion: pkg.crossplane.io/v1beta1
kind: Function
metadata:
  name: function-go-templating
spec:
  package: xpkg.upbound.io/crossplane-contrib/function-go-templating:v0.5.0
---
apiVersion: pkg.crossplane.io/v1beta1
kind: Function
metadata:
  name: function-environment-configs
spec:
  package: xpkg.upbound.io/crossplane-contrib/function-environment-configs:v0.0.7
---
apiVersion: pkg.crossplane.io/v1beta1
kind: Function
metadata:
  name: function-patch-and-transform
spec:
  package: xpkg.upbound.io/crossplane-contrib/function-patch-and-transform:v0.7.0

How can we reproduce it?

Create a complex Composition made up of multiple compositions using go-templating format and it should generate only one of the nested/complex Compositions

What environment did it happen in?

bobh66 commented 1 month ago

@Checksumz The problem is caused by the gotemplating.fn.crossplane.io/composition-resource-name: annotation being set to the same value in both resources. The composition-resource-name MUST be unique for each resource within the composite, otherwise duplicates get overwritten and only the last resource is rendered. You could set the gotemplating.fn.crossplane.io/composition-resource-name: to a for one resource and b for the other and it should work just fine.

Checksumz commented 1 month ago

thanks so much @bobh66, I can't believe i missed that. Closing the issue now.