Peefy
commented
2 months ago Sorry, I've fixed it in v0.9.3. Thanks for the feedback.
Open vfarcic opened 2 months ago
Peefy
commented
2 months ago Sorry, I've fixed it in v0.9.3. Thanks for the feedback.
vfarcic
commented
2 months ago It's still not working. With v0.9.3 I'm getting:
cannot compose resources: pipeline step "apps" returned a fatal result: cannot process xr and state with the pipeline output in *v1beta1.RunFunctionResponse: duplicate resource names a-team-aks-ns-dev found, when returning multiple resources, you need to set different metadata.name or matadata.annotations."krm.kcl.dev/composition-resource-name" to distinguish between different resources in the composition functions.If I downgrade it to v0.9.0 it goes back to "normal" (it works).
Can you reopen this issue? Is there anything I can do to help resolve it?
Peefy
commented
2 months ago Thank you, I've reopened this issue. 🤔 Let me investigate it.
Peefy
commented
2 months ago Hello @vfarcic Can you provide some more detailed information?
I've tried the xr.yaml and compositions.yaml as follows at https://github.com/vfarcic/crossplane-kubernetes/tree/main/package
It works well for me. 🤔 And if I repeatedly set oxr.spec.parameters.namespaces, it will also get the expected resource duplication error.
apiVersion: devopstoolkitseries.com/v1alpha1
kind: CompositeCluster
metadata:
labels:
crossplane.io/claim-name: a-team-eks
spec:
claimRef:
apiVersion: devopstoolkitseries.com/v1alpha1
kind: ClusterClaim
name: a-team-eks
compositionRef:
name: cluster-aws
compositionSelector:
matchLabels:
cluster: eks
provider: aws
id: a-team-eks
parameters:
nodeSize: medium
minNodeCount: 1
usage:
enabled: true
apps:
traefik:
enabled: true
argocd:
enabled: true
host: argocd.44.194.69.17.nip.io
repoURL: https://github.com/vfarcic/crossplane-kubernetes-gitops
dapr:
enabled: true
namespaces:
- dev
- production
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
labels:
cluster: gke
provider: google
name: cluster-google
spec:
compositeTypeRef:
apiVersion: devopstoolkitseries.com/v1alpha1
kind: CompositeCluster
mode: Pipeline
pipeline:
- step: google
functionRef:
name: crossplane-contrib-function-kcl
input:
apiVersion: krm.kcl.dev/v1alpha1
kind: KCLRun
spec:
source: |
oxr = option("params").oxr
ocds = option("params").ocds
_metadata = lambda suffix: str -> any {
{
name = oxr.spec.id
annotations = {
"crossplane.io/external-name" = oxr.spec.id
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-" + suffix
}
}
}
items = [{
apiVersion = "container.gcp.upbound.io/v1beta2"
kind = "Cluster"
metadata = _metadata("cluster")
spec = {
forProvider = {
deletionProtection = False
location = "us-east1"
minMasterVersion = oxr.spec.parameters.version
initialNodeCount = 1
removeDefaultNodePool = True
clusterAutoscaling.autoProvisioningDefaults.management = {
autoRepair = True
autoUpgrade = True
}
}
writeConnectionSecretToRef = {
name = oxr.spec.id + "-cluster"
namespace = oxr.spec.claimRef.namespace
}
}
}, {
apiVersion = "container.gcp.upbound.io/v1beta2"
kind = "NodePool"
metadata = _metadata("nodepool")
spec.forProvider = {
version = oxr.spec.parameters.version
initialNodeCount = oxr.spec.parameters.minNodeCount
nodeLocations = ["us-east1-b", "us-east1-c", "us-east1-d"]
clusterSelector.matchControllerRef = True
nodeConfig = {
if oxr.spec.parameters.nodeSize == "small":
machineType = "e2-standard-2"
elif oxr.spec.parameters.nodeSize == "medium":
machineType = "e2-standard-4"
else:
machineType = "e2-standard-16"
oauthScopes = ["https://www.googleapis.com/auth/cloud-platform"]
taint = [{
key = "node.cilium.io/agent-not-ready"
value = "true"
effect = "NO_EXECUTE"
}]
}
autoscaling = {
minNodeCount = oxr.spec.parameters.minNodeCount
maxNodeCount = 10
}
management = {
autoRepair = True
autoUpgrade = True
}
}
}, {
**oxr
status.clusterName = oxr.spec.id
if oxr.spec.id + "-cluster" in ocds:
status.controlPlaneStatus = ocds[oxr.spec.id + "-cluster"].Resource.status.conditions[0].reason
status.field1 = ocds[oxr.spec.id + "-cluster"].Resource.status.atProvider.clusterIpv4Cidr
if oxr.spec.id + "-nodepool" in ocds:
status.nodePoolStatus = ocds[oxr.spec.id + "-nodepool"].Resource.status.conditions[0].reason
}]
- step: apps
functionRef:
name: crossplane-contrib-function-kcl
input:
apiVersion: krm.kcl.dev/v1alpha1
kind: KCLRun
spec:
source: |
crossplane = "1.14.5"
argocd = "3.35.4"
dapr = "1.12.4"
traefik = "26.0.0"
dynatraceOperator = "0.15.0"
dynatraceDashboard = "0.2.2"
externalSecrets = "0.9.11"
cilium = "1.14.2"
openFunctionUrl = "https://openfunction.github.io/charts/openfunction-v1.2.0-v0.7.0.tgz"
oxr = option("params").oxr
ocds = option("params").ocds
_metadata = lambda suffix: str -> any {
{
name = oxr.spec.id
annotations = {
"crossplane.io/external-name" = oxr.spec.id
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-" + suffix
}
}
}
schema usage:
_nameSuffix: str
_kind: str = "Object"
apiVersion = "apiextensions.crossplane.io/v1alpha1"
kind = "Usage"
metadata = {
name = oxr.spec.id + "-" + _nameSuffix + "-usage"
annotations = {
"crossplane.io/external-name" = oxr.spec.id + "-" + _nameSuffix + "-usage"
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-" + _nameSuffix + "-usage"
}
}
spec = {
of = {
apiVersion = "container.gcp.upbound.io/v1beta2"
kind = "Cluster"
resourceRef.name = oxr.spec.id
}
by = {
if _kind == "Object":
apiVersion = "kubernetes.crossplane.io/v1alpha2"
elif _kind == "Release":
apiVersion = "helm.crossplane.io/v1beta1"
kind = _kind
resourceRef.name = oxr.spec.id + "-" + _nameSuffix
}
}
schema chart:
_name: str
_chartName?: str
_chartRepository?: str
_chartVersion?: str
_chartUrl?: str
_namespace: str
_values?: any
_providerConfigRefName?: str
apiVersion = "helm.crossplane.io/v1beta1"
kind = "Release"
metadata = {
name = oxr.spec.id + "-app-" + _name
annotations = {
"crossplane.io/external-name" = _name
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-app-" + _name
}
}
spec = {
forProvider = {
chart = {
if _chartName:
name = _chartName
else:
name = _name
if _chartRepository:
repository = _chartRepository
if _chartVersion:
version = _chartVersion
if _chartUrl:
url = _chartUrl
}
if _values:
values = _values
namespace = _namespace
}
rollbackLimit = 3
if _providerConfigRefName:
providerConfigRef.name = _providerConfigRefName
else:
providerConfigRef.name = oxr.spec.id
}
schema object:
_name: str
_externalName?: str
_manifest: any
_references?: []
apiVersion = "kubernetes.crossplane.io/v1alpha2"
kind = "Object"
metadata = {
name = oxr.spec.id + "-app-" + _name
annotations = {
if _externalName:
"crossplane.io/external-name" = _externalName
else:
"crossplane.io/external-name" = _name
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-app-" + _name
}
}
spec = {
if _references:
references = _references
forProvider.manifest = _manifest
providerConfigRef.name = oxr.spec.id
}
_items = [
{
apiVersion = "helm.crossplane.io/v1beta1"
kind = "ProviderConfig"
metadata = {
name = oxr.spec.id
annotations = {
"krm.kcl.dev/ready": "True"
"crossplane.io/external-name" = oxr.spec.id
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-config-helm"
}
}
spec = {
credentials = {
secretRef = {
namespace = oxr.spec.claimRef.namespace
key = "kubeconfig"
name = oxr.spec.id + "-cluster"
}
source = "Secret"
}
if oxr.spec.compositionSelector.matchLabels.provider == "google":
identity = {
type = "GoogleApplicationCredentials"
source = "Secret"
secretRef = {
name = "gcp-creds"
namespace = "crossplane-system"
key = "creds"
}
}
}
}
{
apiVersion = "helm.crossplane.io/v1beta1"
kind = "ProviderConfig"
metadata = {
name = oxr.spec.id + "-local"
annotations = {
"krm.kcl.dev/ready": "True"
"crossplane.io/external-name" = oxr.spec.id
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-config-helm-local"
}
}
spec.credentials.source = "InjectedIdentity"
}
if oxr.spec.compositionSelector.matchLabels.provider != "aws":
{
apiVersion = "helm.crossplane.io/v1beta1"
kind = "Release"
metadata = {
name = oxr.spec.id + "-cilium"
annotations = {
"crossplane.io/external-name" = "cilium"
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-app-cilium"
}
}
spec = {
forProvider = {
chart = {
name = "cilium"
repository = "https://helm.cilium.io"
version = cilium
}
set = [
if oxr.spec.compositionSelector.matchLabels.provider == "google":
{name = "nodeinit.enabled", value = "true"}
{name = "nodeinit.reconfigureKubelet", value = "true"}
{name = "nodeinit.removeCbrBridge", value = "true"}
{name = "cni.binPath", value = "/home/kubernetes/bin"}
{name = "gke.enabled", value = "true"}
{name = "ipam.mode", value = "kubernetes"}
{
name = "ipv4NativeRoutingCIDR"
if oxr.spec.id + "-nodepool" in ocds:
value = oxr.status.field1
},
{name = "authentication.mutual.spire.enabled", value = "true"}
{name = "authentication.mutual.spire.install.enabled", value = "true"}
if oxr.spec.compositionSelector.matchLabels.provider == "azure":
{name = "aksbyocni.enabled", value = "true"}
{name = "nodeinit.enabled", value = "true"}
{name = "authentication.mutual.spire.enabled", value = "true"}
{name = "authentication.mutual.spire.install.enabled", value = "true"}
]
namespace = "kube-system"
}
rollbackLimit = 3
providerConfigRef.name = oxr.spec.id
}
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "cilium", _kind = "Release" }
{
apiVersion = "kubernetes.crossplane.io/v1alpha1"
kind = "ProviderConfig"
metadata = {
name = oxr.spec.id
annotations = {
"krm.kcl.dev/ready": "True"
"crossplane.io/external-name" = oxr.spec.id
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-config-kubernetes"
}
}
spec = {
credentials = {
secretRef = {
key = "kubeconfig"
name = oxr.spec.id + "-cluster"
namespace = oxr.spec.claimRef.namespace
}
source: "Secret"
}
if oxr.spec.compositionSelector.matchLabels.provider == "google":
identity = {
type = "GoogleApplicationCredentials"
source = "Secret"
secretRef = {
name = "gcp-creds"
namespace = "crossplane-system"
key = "creds"
}
}
}
}
if oxr.spec.parameters?.apps?.crossplane?.enabled:
chart {
_name = "crossplane"
_chartRepository = "https://charts.crossplane.io/stable"
_chartVersion = crossplane
_namespace = "crossplane-system"
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "crossplane", _kind = "Release" }
if oxr.spec.parameters?.apps?.argocd?.enabled:
chart {
_name = "argo-cd"
_chartRepository = "https://argoproj.github.io/argo-helm"
_chartVersion = argocd
_namespace = "argocd"
_values = {
global.domain = oxr.spec.parameters.apps.argocd.host
configs = {
secret = {
argocdServerAdminPassword = "$2a$10$m3eTlEdRen0nS86c5Zph5u/bDFQMcWZYdG3NVdiyaACCqoxLJaz16"
argocdServerAdminPasswordMtime = "2021-11-08T15:04:05Z"
}
cm = {
"application.resourceTrackingMethod" = "annotation"
"timeout.reconciliation" = "60s"
}
params = {
"server.insecure" = True
}
}
server = {
if oxr.spec.parameters?.apps?.traefik?.enabled:
ingress = {
enabled = True
ingressClassName = "traefik"
}
extraArgs = ["--insecure"]
}
}
}
object {
_name = "argo-cd-app"
_manifest = {
apiVersion = "argoproj.io/v1alpha1"
kind = "Application"
metadata = {
name = "apps"
namespace = "argocd"
finalizers = ["resources-finalizer.argocd.argoproj.io"]
}
spec = {
project = "default"
source = {
repoURL = oxr.spec.parameters.apps.argocd.repoURL
targetRevision = "HEAD"
path = oxr.spec.parameters.apps.argocd.sourcePath
}
destination = {
server = "https://kubernetes.default.svc"
namespace = oxr.spec.parameters.apps.argocd.destinationNamespace
}
syncPolicy.automated = {
selfHeal = True
prune = True
allowEmpty = True
}
}
}
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "argo-cd", _kind = "Release" }
usage { _nameSuffix = "argo-cd-app" }
if oxr.spec.parameters?.apps?.openfunction?.enabled:
chart {
_name = "openfunction"
_chartUrl = openFunctionUrl
_namespace = "openfunction"
_values = {
revisionController.enable = True
}
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "openfunction", _kind = "Release" }
if oxr.spec.parameters?.apps?.dapr?.enabled:
chart {
_name = "dapr"
_chartRepository = "https://dapr.github.io/helm-charts/"
_chartVersion = dapr
_namespace = "dapr-system"
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "dapr", _kind = "Release" }
if oxr.spec.parameters?.apps?.traefik?.enabled:
chart {
_name = "traefik"
_chartRepository = "https://helm.traefik.io/traefik"
_chartVersion = traefik
_namespace = "traefik"
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "traefik", _kind = "Release" }
if oxr.spec.parameters?.apps?.dynatrace?.enabled:
chart {
_name = "dynatrace-operator"
_chartRepository = "https://raw.githubusercontent.com/Dynatrace/dynatrace-operator/main/config/helm/repos/stable"
_chartVersion = dynatraceOperator
_namespace = "dynatrace"
_values = {
installCRD = True
csidriver.enabled = True
}
}
object {
_name = "dynakube"
_manifest = {
apiVersion = "dynatrace.com/v1beta1"
kind = "DynaKube"
metadata = {
name = oxr.spec.id
namespace = "dynatrace"
annotations = {
"feature.dynatrace.com/k8s-app-enabled" = "true"
}
}
spec = {
apiUrl = oxr.spec.parameters.apps.dynatrace.apiUrl
oneAgent.cloudNativeFullStack.image = ""
activeGate = {
capabilities = [
"kubernetes-monitoring"
"routing"
"metrics-ingest"
"dynatrace-api"
]
image = ""
resources = {
requests = {
cpu = "500m"
memory = "512Mi"
}
limits = {
cpu = "1000m"
memory = "1.5Gi"
}
}
}
}
}
}
chart {
_name = "dynatrace-dashboard"
_chartName = "kubernetes-cluster"
_chartRepository = "https://katharinasick.github.io/crossplane-observability-demo-dynatrace"
_chartVersion = dynatraceDashboard
_namespace = "dynatrace"
_values = {
oauthCredentialsSecretName = oxr.spec.parameters.apps.dynatrace.oathCredentialsSecretName
cluster = oxr.spec.id
dashboards = {
clusterOverview.enabled = True
crossplaneMetrics.enabled = False
}
}
_providerConfigRefName = oxr.spec.id + "-local"
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "dynatrace-operator", _kind = "Release" }
usage { _nameSuffix = "dynakube" }
usage { _nameSuffix = "dynatrace-dashboard", _kind = "Release" }
if oxr.spec.parameters?.apps?.externalSecrets?.enabled:
chart {
_name = "external-secrets"
_chartRepository = "https://charts.external-secrets.io"
_chartVersion = externalSecrets
_namespace = "external-secrets"
_values = {
installCRDs = True
}
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "external-secrets", _kind = "Release" }
if oxr.spec.parameters?.apps?.externalSecrets?.enabled and oxr.spec.parameters.apps?.externalSecrets?.store:
object {
_name = "secret-store"
_externalName = oxr.spec.compositionSelector.matchLabels.provider
_manifest = {
apiVersion = "external-secrets.io/v1beta1"
kind = "ClusterSecretStore"
metadata.name = oxr.spec.compositionSelector.matchLabels.provider
if oxr.spec.compositionSelector.matchLabels.provider == "google":
spec.provider.gcpsm.auth.secretRef.secretAccessKeySecretRef = {
name: "gcp-creds"
key: oxr.spec.parameters.apps.externalSecrets.googleCredentialsKey
namespace: oxr.spec.parameters.creds.namespace
}
elif oxr.spec.compositionSelector.matchLabels.provider == "azure":
spec.provider.azurekv = {
authType = "ManagedIdentity"
vaultUrl = oxr.spec.parameters.apps.externalSecrets.azureVaultUrl
}
elif oxr.spec.compositionSelector.matchLabels.provider == "aws":
spec.provider.aws = {
service = "SecretsManager"
region = "us-east-1"
auth.secretRef = {
accessKeyIDSecretRef = {
name = oxr.spec.parameters.creds.name
key = oxr.spec.parameters.apps.externalSecrets.awsAccessKeyIDKey
namespace = oxr.spec.parameters.creds.namespace
}
secretAccessKeySecretRef = {
name = oxr.spec.parameters.creds.name
key = oxr.spec.parameters.apps.externalSecrets.awsSecretAccessKeyKey
namespace = oxr.spec.parameters.creds.namespace
}
}
}
}
if oxr.spec.compositionSelector.matchLabels.provider == "google":
_references = [{
patchesFrom = {
apiVersion = "gcp.upbound.io/v1beta1"
kind = "ProviderConfig"
name = "default"
fieldPath = "spec.projectID"
}
toFieldPath = "spec.provider.gcpsm.projectID"
}]
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "secret-store" }
]
if oxr.spec.parameters?.apps?.externalSecrets?.secrets:
_items += [{
apiVersion = "kubernetes.crossplane.io/v1alpha2"
kind = "Object"
metadata = {
name = oxr.spec.id + "-secret-" + _secret.toSecret
annotations = {
"crossplane.io/external-name" = _secret.toSecret
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-secret-" + _secret.toSecret
}
}
spec = {
forProvider.manifest = {
apiVersion = "external-secrets.io/v1beta1"
kind = "ExternalSecret"
metadata = {
name: _secret.toSecret
namespace: _secret.toNamespace
}
spec = {
refreshInterval = "1h"
secretStoreRef = {
kind: "ClusterSecretStore"
name: oxr.spec.compositionSelector.matchLabels.provider
}
target = {
name = _secret.toSecret
creationPolicy = "Owner"
template.type = _secret.type
}
dataFrom = [{ extract.key = _secret.fromSecret }]
}
}
providerConfigRef.name = oxr.spec.id
}
} for _secret in oxr.spec.parameters.apps.externalSecrets.secrets ]
if oxr.spec.parameters?.usage?.enabled:
_items += [ usage {
_nameSuffix = "secret-" + _secret.toSecret
} for _secret in oxr.spec.parameters.apps.externalSecrets.secrets ]
if oxr.spec.parameters?.namespaces:
_items += [{
apiVersion = "kubernetes.crossplane.io/v1alpha2"
kind = "Object"
metadata = {
name = oxr.spec.id + "-ns-" + _namespace
annotations = {
"crossplane.io/external-name" = _namespace
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-ns-" + _namespace
}
}
spec = {
forProvider.manifest = {
apiVersion = "v1"
kind = "Namespace"
metadata.name = _namespace
}
deletionPolicy = "Orphan"
providerConfigRef.name = oxr.spec.id
}
} for _namespace in oxr.spec.parameters.namespaces ]
if oxr.spec.parameters?.creds:
_items += [
{
apiVersion = "kubernetes.crossplane.io/v1alpha2"
kind = "Object"
metadata = {
name = oxr.spec.id + "-creds"
annotations = {
"crossplane.io/external-name" = oxr.spec.parameters.creds.name
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-creds"
}
}
spec = {
references = [{
patchesFrom = {
apiVersion = "v1"
kind = "Secret"
name: oxr.spec.parameters.creds.name
namespace: oxr.spec.parameters.creds.namespace
fieldPath: "data." + _credReference
}
toFieldPath: "data." + _credReference
} for _credReference in oxr.spec.parameters.creds.keys]
forProvider = {
manifest = {
apiVersion = "v1"
kind = "Secret"
metadata = {
name = oxr.spec.parameters.creds.name
namespace = oxr.spec.parameters.creds.namespace
}
}
}
providerConfigRef.name = oxr.spec.id
}
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "creds" }
]
items = _items
writeConnectionSecretsToNamespace: crossplane-system
---
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
labels:
cluster: aks
provider: azure
name: cluster-azure
spec:
compositeTypeRef:
apiVersion: devopstoolkitseries.com/v1alpha1
kind: CompositeCluster
mode: Pipeline
pipeline:
- step: azure
functionRef:
name: crossplane-contrib-function-kcl
input:
apiVersion: krm.kcl.dev/v1alpha1
kind: KCLRun
spec:
source: |
oxr = option("params").oxr
ocds = option("params").ocds
_metadata = lambda suffix: str -> any {
{
name = oxr.spec.id
annotations = {
"crossplane.io/external-name" = oxr.spec.id
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-" + suffix
}
}
}
items = [{
apiVersion = "azure.upbound.io/v1beta1"
kind = "ResourceGroup"
metadata = _metadata("resource-group")
spec = {
forProvider.location = "eastus"
initProvider = {}
}
}, {
apiVersion = "containerservice.azure.upbound.io/v1beta1"
kind = "KubernetesCluster"
metadata = _metadata("cluster")
spec = {
forProvider: {
resourceGroupName = oxr.spec.id
kubernetesVersion = oxr.spec.parameters.version
location = "eastus"
dnsPrefix = "dot"
defaultNodePool = [{
name = oxr.spec.id
if oxr.spec.parameters.nodeSize == "small":
vmSize = "Standard_D2_v2"
elif oxr.spec.parameters.nodeSize == "medium":
vmSize = "Standard_D3_v2"
else:
vmSize = "Standard_D4_v2"
minCount = oxr.spec.parameters.minNodeCount
maxCount = 10
enableAutoScaling = True
}]
identity = [{type = "SystemAssigned"}]
networkProfile = [{networkPlugin = "none"}]
}
writeConnectionSecretToRef = {
name = oxr.spec.id + "-cluster"
namespace = oxr.spec.claimRef.namespace
}
}
}, {
**oxr
status.clusterName = oxr.spec.id
if oxr.spec.id + "-cluster" in ocds:
status.controlPlaneStatus = ocds[oxr.spec.id + "-cluster"].Resource.status.conditions[0].reason
status.nodePoolStatus = ocds[oxr.spec.id + "-cluster"].Resource.status.conditions[0].reason
}]
- step: apps
functionRef:
name: crossplane-contrib-function-kcl
input:
apiVersion: krm.kcl.dev/v1alpha1
kind: KCLRun
spec:
source: |
crossplane = "1.14.5"
argocd = "3.35.4"
dapr = "1.12.4"
traefik = "26.0.0"
dynatraceOperator = "0.15.0"
dynatraceDashboard = "0.2.2"
externalSecrets = "0.9.11"
cilium = "1.14.2"
openFunctionUrl = "https://openfunction.github.io/charts/openfunction-v1.2.0-v0.7.0.tgz"
oxr = option("params").oxr
ocds = option("params").ocds
_metadata = lambda suffix: str -> any {
{
name = oxr.spec.id
annotations = {
"crossplane.io/external-name" = oxr.spec.id
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-" + suffix
}
}
}
schema usage:
_nameSuffix: str
_kind: str = "Object"
apiVersion = "apiextensions.crossplane.io/v1alpha1"
kind = "Usage"
metadata = {
name = oxr.spec.id + "-" + _nameSuffix + "-usage"
annotations = {
"crossplane.io/external-name" = oxr.spec.id + "-" + _nameSuffix + "-usage"
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-" + _nameSuffix + "-usage"
}
}
spec = {
of = {
apiVersion = "containerservice.azure.upbound.io/v1beta1"
kind = "KubernetesCluster"
resourceRef.name = oxr.spec.id
}
by = {
if _kind == "Object":
apiVersion = "kubernetes.crossplane.io/v1alpha2"
elif _kind == "Release":
apiVersion = "helm.crossplane.io/v1beta1"
kind = _kind
resourceRef.name = oxr.spec.id + "-" + _nameSuffix
}
}
schema chart:
_name: str
_chartName?: str
_chartRepository?: str
_chartVersion?: str
_chartUrl?: str
_namespace: str
_values?: any
_providerConfigRefName?: str
apiVersion = "helm.crossplane.io/v1beta1"
kind = "Release"
metadata = {
name = oxr.spec.id + "-app-" + _name
annotations = {
"crossplane.io/external-name" = _name
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-app-" + _name
}
}
spec = {
forProvider = {
chart = {
if _chartName:
name = _chartName
else:
name = _name
if _chartRepository:
repository = _chartRepository
if _chartVersion:
version = _chartVersion
if _chartUrl:
url = _chartUrl
}
if _values:
values = _values
namespace = _namespace
}
rollbackLimit = 3
if _providerConfigRefName:
providerConfigRef.name = _providerConfigRefName
else:
providerConfigRef.name = oxr.spec.id
}
schema object:
_name: str
_externalName?: str
_manifest: any
_references?: []
apiVersion = "kubernetes.crossplane.io/v1alpha2"
kind = "Object"
metadata = {
name = oxr.spec.id + "-app-" + _name
annotations = {
if _externalName:
"crossplane.io/external-name" = _externalName
else:
"crossplane.io/external-name" = _name
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-app-" + _name
}
}
spec = {
if _references:
references = _references
forProvider.manifest = _manifest
providerConfigRef.name = oxr.spec.id
}
_items = [
{
apiVersion = "helm.crossplane.io/v1beta1"
kind = "ProviderConfig"
metadata = {
name = oxr.spec.id
annotations = {
"krm.kcl.dev/ready": "True"
"crossplane.io/external-name" = oxr.spec.id
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-config-helm"
}
}
spec = {
credentials = {
secretRef = {
namespace = oxr.spec.claimRef.namespace
key = "kubeconfig"
name = oxr.spec.id + "-cluster"
}
source = "Secret"
}
if oxr.spec.compositionSelector.matchLabels.provider == "google":
identity = {
type = "GoogleApplicationCredentials"
source = "Secret"
secretRef = {
name = "gcp-creds"
namespace = "crossplane-system"
key = "creds"
}
}
}
}
{
apiVersion = "helm.crossplane.io/v1beta1"
kind = "ProviderConfig"
metadata = {
name = oxr.spec.id + "-local"
annotations = {
"krm.kcl.dev/ready": "True"
"crossplane.io/external-name" = oxr.spec.id
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-config-helm-local"
}
}
spec.credentials.source = "InjectedIdentity"
}
if oxr.spec.compositionSelector.matchLabels.provider != "aws":
{
apiVersion = "helm.crossplane.io/v1beta1"
kind = "Release"
metadata = {
name = oxr.spec.id + "-cilium"
annotations = {
"crossplane.io/external-name" = "cilium"
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-app-cilium"
}
}
spec = {
forProvider = {
chart = {
name = "cilium"
repository = "https://helm.cilium.io"
version = cilium
}
set = [
if oxr.spec.compositionSelector.matchLabels.provider == "google":
{name = "nodeinit.enabled", value = "true"}
{name = "nodeinit.reconfigureKubelet", value = "true"}
{name = "nodeinit.removeCbrBridge", value = "true"}
{name = "cni.binPath", value = "/home/kubernetes/bin"}
{name = "gke.enabled", value = "true"}
{name = "ipam.mode", value = "kubernetes"}
{
name = "ipv4NativeRoutingCIDR"
if oxr.spec.id + "-nodepool" in ocds:
value = oxr.status.field1
},
{name = "authentication.mutual.spire.enabled", value = "true"}
{name = "authentication.mutual.spire.install.enabled", value = "true"}
if oxr.spec.compositionSelector.matchLabels.provider == "azure":
{name = "aksbyocni.enabled", value = "true"}
{name = "nodeinit.enabled", value = "true"}
{name = "authentication.mutual.spire.enabled", value = "true"}
{name = "authentication.mutual.spire.install.enabled", value = "true"}
]
namespace = "kube-system"
}
rollbackLimit = 3
providerConfigRef.name = oxr.spec.id
}
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "cilium", _kind = "Release" }
{
apiVersion = "kubernetes.crossplane.io/v1alpha1"
kind = "ProviderConfig"
metadata = {
name = oxr.spec.id
annotations = {
"krm.kcl.dev/ready": "True"
"crossplane.io/external-name" = oxr.spec.id
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-config-kubernetes"
}
}
spec = {
credentials = {
secretRef = {
key = "kubeconfig"
name = oxr.spec.id + "-cluster"
namespace = oxr.spec.claimRef.namespace
}
source: "Secret"
}
if oxr.spec.compositionSelector.matchLabels.provider == "google":
identity = {
type = "GoogleApplicationCredentials"
source = "Secret"
secretRef = {
name = "gcp-creds"
namespace = "crossplane-system"
key = "creds"
}
}
}
}
if oxr.spec.parameters?.apps?.crossplane?.enabled:
chart {
_name = "crossplane"
_chartRepository = "https://charts.crossplane.io/stable"
_chartVersion = crossplane
_namespace = "crossplane-system"
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "crossplane", _kind = "Release" }
if oxr.spec.parameters?.apps?.argocd?.enabled:
chart {
_name = "argo-cd"
_chartRepository = "https://argoproj.github.io/argo-helm"
_chartVersion = argocd
_namespace = "argocd"
_values = {
global.domain = oxr.spec.parameters.apps.argocd.host
configs = {
secret = {
argocdServerAdminPassword = "$2a$10$m3eTlEdRen0nS86c5Zph5u/bDFQMcWZYdG3NVdiyaACCqoxLJaz16"
argocdServerAdminPasswordMtime = "2021-11-08T15:04:05Z"
}
cm = {
"application.resourceTrackingMethod" = "annotation"
"timeout.reconciliation" = "60s"
}
params = {
"server.insecure" = True
}
}
server = {
if oxr.spec.parameters?.apps?.traefik?.enabled:
ingress = {
enabled = True
ingressClassName = "traefik"
}
extraArgs = ["--insecure"]
}
}
}
object {
_name = "argo-cd-app"
_manifest = {
apiVersion = "argoproj.io/v1alpha1"
kind = "Application"
metadata = {
name = "apps"
namespace = "argocd"
finalizers = ["resources-finalizer.argocd.argoproj.io"]
}
spec = {
project = "default"
source = {
repoURL = oxr.spec.parameters.apps.argocd.repoURL
targetRevision = "HEAD"
path = oxr.spec.parameters.apps.argocd.sourcePath
}
destination = {
server = "https://kubernetes.default.svc"
namespace = oxr.spec.parameters.apps.argocd.destinationNamespace
}
syncPolicy.automated = {
selfHeal = True
prune = True
allowEmpty = True
}
}
}
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "argo-cd", _kind = "Release" }
usage { _nameSuffix = "argo-cd-app" }
if oxr.spec.parameters?.apps?.openfunction?.enabled:
chart {
_name = "openfunction"
_chartUrl = openFunctionUrl
_namespace = "openfunction"
_values = {
revisionController.enable = True
}
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "openfunction", _kind = "Release" }
if oxr.spec.parameters?.apps?.dapr?.enabled:
chart {
_name = "dapr"
_chartRepository = "https://dapr.github.io/helm-charts/"
_chartVersion = dapr
_namespace = "dapr-system"
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "dapr", _kind = "Release" }
if oxr.spec.parameters?.apps?.traefik?.enabled:
chart {
_name = "traefik"
_chartRepository = "https://helm.traefik.io/traefik"
_chartVersion = traefik
_namespace = "traefik"
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "traefik", _kind = "Release" }
if oxr.spec.parameters?.apps?.dynatrace?.enabled:
chart {
_name = "dynatrace-operator"
_chartRepository = "https://raw.githubusercontent.com/Dynatrace/dynatrace-operator/main/config/helm/repos/stable"
_chartVersion = dynatraceOperator
_namespace = "dynatrace"
_values = {
installCRD = True
csidriver.enabled = True
}
}
object {
_name = "dynakube"
_manifest = {
apiVersion = "dynatrace.com/v1beta1"
kind = "DynaKube"
metadata = {
name = oxr.spec.id
namespace = "dynatrace"
annotations = {
"feature.dynatrace.com/k8s-app-enabled" = "true"
}
}
spec = {
apiUrl = oxr.spec.parameters.apps.dynatrace.apiUrl
oneAgent.cloudNativeFullStack.image = ""
activeGate = {
capabilities = [
"kubernetes-monitoring"
"routing"
"metrics-ingest"
"dynatrace-api"
]
image = ""
resources = {
requests = {
cpu = "500m"
memory = "512Mi"
}
limits = {
cpu = "1000m"
memory = "1.5Gi"
}
}
}
}
}
}
chart {
_name = "dynatrace-dashboard"
_chartName = "kubernetes-cluster"
_chartRepository = "https://katharinasick.github.io/crossplane-observability-demo-dynatrace"
_chartVersion = dynatraceDashboard
_namespace = "dynatrace"
_values = {
oauthCredentialsSecretName = oxr.spec.parameters.apps.dynatrace.oathCredentialsSecretName
cluster = oxr.spec.id
dashboards = {
clusterOverview.enabled = True
crossplaneMetrics.enabled = False
}
}
_providerConfigRefName = oxr.spec.id + "-local"
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "dynatrace-operator", _kind = "Release" }
usage { _nameSuffix = "dynakube" }
usage { _nameSuffix = "dynatrace-dashboard", _kind = "Release" }
if oxr.spec.parameters?.apps?.externalSecrets?.enabled:
chart {
_name = "external-secrets"
_chartRepository = "https://charts.external-secrets.io"
_chartVersion = externalSecrets
_namespace = "external-secrets"
_values = {
installCRDs = True
}
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "external-secrets", _kind = "Release" }
if oxr.spec.parameters?.apps?.externalSecrets?.enabled and oxr.spec.parameters.apps?.externalSecrets?.store:
object {
_name = "secret-store"
_externalName = oxr.spec.compositionSelector.matchLabels.provider
_manifest = {
apiVersion = "external-secrets.io/v1beta1"
kind = "ClusterSecretStore"
metadata.name = oxr.spec.compositionSelector.matchLabels.provider
if oxr.spec.compositionSelector.matchLabels.provider == "google":
spec.provider.gcpsm.auth.secretRef.secretAccessKeySecretRef = {
name: "gcp-creds"
key: oxr.spec.parameters.apps.externalSecrets.googleCredentialsKey
namespace: oxr.spec.parameters.creds.namespace
}
elif oxr.spec.compositionSelector.matchLabels.provider == "azure":
spec.provider.azurekv = {
authType = "ManagedIdentity"
vaultUrl = oxr.spec.parameters.apps.externalSecrets.azureVaultUrl
}
elif oxr.spec.compositionSelector.matchLabels.provider == "aws":
spec.provider.aws = {
service = "SecretsManager"
region = "us-east-1"
auth.secretRef = {
accessKeyIDSecretRef = {
name = oxr.spec.parameters.creds.name
key = oxr.spec.parameters.apps.externalSecrets.awsAccessKeyIDKey
namespace = oxr.spec.parameters.creds.namespace
}
secretAccessKeySecretRef = {
name = oxr.spec.parameters.creds.name
key = oxr.spec.parameters.apps.externalSecrets.awsSecretAccessKeyKey
namespace = oxr.spec.parameters.creds.namespace
}
}
}
}
if oxr.spec.compositionSelector.matchLabels.provider == "google":
_references = [{
patchesFrom = {
apiVersion = "gcp.upbound.io/v1beta1"
kind = "ProviderConfig"
name = "default"
fieldPath = "spec.projectID"
}
toFieldPath = "spec.provider.gcpsm.projectID"
}]
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "secret-store" }
]
if oxr.spec.parameters?.apps?.externalSecrets?.secrets:
_items += [{
apiVersion = "kubernetes.crossplane.io/v1alpha2"
kind = "Object"
metadata = {
name = oxr.spec.id + "-secret-" + _secret.toSecret
annotations = {
"crossplane.io/external-name" = _secret.toSecret
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-secret-" + _secret.toSecret
}
}
spec = {
forProvider.manifest = {
apiVersion = "external-secrets.io/v1beta1"
kind = "ExternalSecret"
metadata = {
name: _secret.toSecret
namespace: _secret.toNamespace
}
spec = {
refreshInterval = "1h"
secretStoreRef = {
kind: "ClusterSecretStore"
name: oxr.spec.compositionSelector.matchLabels.provider
}
target = {
name = _secret.toSecret
creationPolicy = "Owner"
template.type = _secret.type
}
dataFrom = [{ extract.key = _secret.fromSecret }]
}
}
providerConfigRef.name = oxr.spec.id
}
} for _secret in oxr.spec.parameters.apps.externalSecrets.secrets ]
if oxr.spec.parameters?.usage?.enabled:
_items += [ usage {
_nameSuffix = "secret-" + _secret.toSecret
} for _secret in oxr.spec.parameters.apps.externalSecrets.secrets ]
if oxr.spec.parameters?.namespaces:
_items += [{
apiVersion = "kubernetes.crossplane.io/v1alpha2"
kind = "Object"
metadata = {
name = oxr.spec.id + "-ns-" + _namespace
annotations = {
"crossplane.io/external-name" = _namespace
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-ns-" + _namespace
}
}
spec = {
forProvider.manifest = {
apiVersion = "v1"
kind = "Namespace"
metadata.name = _namespace
}
deletionPolicy = "Orphan"
providerConfigRef.name = oxr.spec.id
}
} for _namespace in oxr.spec.parameters.namespaces ]
if oxr.spec.parameters?.creds:
_items += [
{
apiVersion = "kubernetes.crossplane.io/v1alpha2"
kind = "Object"
metadata = {
name = oxr.spec.id + "-creds"
annotations = {
"crossplane.io/external-name" = oxr.spec.parameters.creds.name
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-creds"
}
}
spec = {
references = [{
patchesFrom = {
apiVersion = "v1"
kind = "Secret"
name: oxr.spec.parameters.creds.name
namespace: oxr.spec.parameters.creds.namespace
fieldPath: "data." + _credReference
}
toFieldPath: "data." + _credReference
} for _credReference in oxr.spec.parameters.creds.keys]
forProvider = {
manifest = {
apiVersion = "v1"
kind = "Secret"
metadata = {
name = oxr.spec.parameters.creds.name
namespace = oxr.spec.parameters.creds.namespace
}
}
}
providerConfigRef.name = oxr.spec.id
}
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "creds" }
]
items = _items
- step: automatically-detect-ready-composed-resources
functionRef:
name: crossplane-contrib-function-auto-ready
writeConnectionSecretsToNamespace: crossplane-system
---
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
labels:
cluster: eks
provider: aws
name: cluster-aws
spec:
compositeTypeRef:
apiVersion: devopstoolkitseries.com/v1alpha1
kind: CompositeCluster
mode: Pipeline
pipeline:
- step: aws
functionRef:
name: crossplane-contrib-function-kcl
input:
apiVersion: krm.kcl.dev/v1alpha1
kind: KCLRun
spec:
source: |
version = "1.30"
oxr = option("params").oxr
ocds = option("params").ocds
_metadata = lambda name: str, suffix: str, role: str -> any {
{
if name != "":
name = oxr.spec.id + "-" + name
else:
name = oxr.spec.id
annotations = {
"krm.kcl.dev/composition-resource-name" = suffix
}
if role != "":
labels.role = oxr.spec.id + "-" + role
}
}
_items = [{
apiVersion = "eks.aws.upbound.io/v1beta1"
kind = "Cluster"
metadata = _metadata("", "ekscluster", "")
spec.forProvider: {
region = "us-east-1"
bootstrapSelfManagedAddons = True
if oxr.spec.parameters.version:
version = oxr.spec.parameters.version
else:
version = version
roleArnSelector = {
matchControllerRef = True
matchLabels.role = oxr.spec.id + "-controlplane"
}
vpcConfig = [{
endpointPrivateAccess = True
endpointPublicAccess = True
subnetIdSelector.matchControllerRef = True
}]
}
}, {
apiVersion = "eks.aws.upbound.io/v1beta1"
kind = "ClusterAuth"
metadata = _metadata("", "clusterAuth", "")
spec = {
forProvider: {
region = "us-east-1"
clusterNameSelector.matchControllerRef = True
}
writeConnectionSecretToRef = {
name = oxr.spec.id + "-cluster"
namespace = oxr.spec.claimRef.namespace
}
}
}, {
apiVersion = "eks.aws.upbound.io/v1beta1"
kind = "NodeGroup"
metadata = _metadata("", "eksnodegroup", "")
spec = {
forProvider = {
region = "us-east-1"
clusterNameSelector.matchControllerRef = True
nodeRoleArnSelector = {
matchControllerRef = True
matchLabels.role = oxr.spec.id + "-nodegroup"
}
subnetIdSelector.matchControllerRef = True
scalingConfig = [{
minSize = 1
maxSize = 10
desiredSize = 1
minSize = oxr.spec.parameters.minNodeCount
desiredSize = oxr.spec.parameters.minNodeCount
}]
if oxr.spec.parameters.nodeSize == "small":
instanceTypes = ["t3.small"]
elif oxr.spec.parameters.nodeSize == "medium":
instanceTypes = ["t3.medium"]
else:
instanceTypes = ["t3.large"]
}
}
}, {
apiVersion = "ec2.aws.upbound.io/v1beta1"
kind = "VPC"
metadata = _metadata("", "vpc-nodepool", "")
spec = {
forProvider = {
region = "us-east-1"
cidrBlock = "10.0.0.0/16"
enableDnsSupport = True
}
}
}, {
apiVersion: "ec2.aws.upbound.io/v1beta1"
kind: "SecurityGroup"
metadata = _metadata("", "sg-nodepool", "")
spec.forProvider: {
name = oxr.spec.id
description = "Cluster communication with worker nodes"
region = "us-east-1"
vpcIdSelector.matchControllerRef = True
}
}, {
apiVersion = "ec2.aws.upbound.io/v1beta1"
kind = "SecurityGroupRule"
metadata = _metadata("", "securityGroupRule", "")
spec.forProvider: {
description = "I am too lazy to write descriptions"
region = "us-east-1"
type = "egress"
fromPort = 0
toPort = 0
protocol = "-1"
cidrBlocks = ["0.0.0.0/0"]
securityGroupIdSelector.matchControllerRef = True
}
}, {
apiVersion = "ec2.aws.upbound.io/v1beta1"
kind = "InternetGateway"
metadata = _metadata("", "gateway", "")
spec.forProvider = {
region = "us-east-1"
vpcIdSelector.matchControllerRef = True
}
}, {
apiVersion = "ec2.aws.upbound.io/v1beta1"
kind = "RouteTable"
metadata = _metadata("", "routeTable", "")
spec.forProvider: {
region = "us-east-1"
vpcIdSelector.matchControllerRef = True
}
}, {
apiVersion: "ec2.aws.upbound.io/v1beta1"
kind: "Route"
metadata = _metadata("", "route", "")
spec.forProvider: {
region = "us-east-1"
routeTableIdSelector.matchControllerRef = True
destinationCidrBlock = "0.0.0.0/0"
gatewayIdSelector.matchControllerRef = True
}
}, {
apiVersion: "ec2.aws.upbound.io/v1beta1"
kind: "MainRouteTableAssociation"
metadata = _metadata("", "mainRouteTableAssociation", "")
spec.forProvider: {
region = "us-east-1"
routeTableIdSelector.matchControllerRef = True
vpcIdSelector.matchControllerRef = True
}
}, {
apiVersion = "eks.aws.upbound.io/v1beta1"
kind = "Addon"
metadata = _metadata("ebs", "addonEbs", "")
spec.forProvider = {
addonName = "aws-ebs-csi-driver"
region = "us-east-1"
clusterNameSelector.matchControllerRef = True
}
}, {
**oxr
status.clusterName = oxr.spec.id
if oxr.spec.id + "-cluster" in ocds:
status.controlPlaneStatus = ocds[oxr.spec.id + "-cluster"].Resource.status.conditions[0].reason
if oxr.spec.id + "-node-group" in ocds:
status.nodePoolStatus = ocds[oxr.spec.id + "-node-group"].Resource.status.conditions[0].reason
}]
_zones = [
{suffix = "1a", ip = "10.0.0.0/24"}
{suffix = "1b", ip = "10.0.1.0/24"},
{suffix = "1c", ip = "10.0.2.0/24"}
]
_items += [{
apiVersion = "ec2.aws.upbound.io/v1beta1"
kind = "RouteTableAssociation"
metadata = _metadata(_zone.suffix, "routeTableAssociation" + _zone.suffix, "")
spec.forProvider: {
region = "us-east-1"
routeTableIdSelector.matchControllerRef = True
subnetIdSelector = {
matchControllerRef = True
matchLabels = {
zone = "us-east-" + _zone.suffix
access = "public"
}
}
}
} for _zone in _zones]
_items += [{
apiVersion = "ec2.aws.upbound.io/v1beta1"
kind = "Subnet"
metadata = {
name = oxr.spec.id + "-" + _zone.suffix
annotations = {
"krm.kcl.dev/composition-resource-name" = "subnet-nodepool-" + _zone.suffix
}
labels = {
zone = "us-east-" + _zone.suffix
access: "public"
}
}
spec: {
forProvider = {
region = "us-east-1"
availabilityZone = "us-east-" + _zone.suffix
cidrBlock = _zone.ip
vpcIdSelector.matchControllerRef = True
mapPublicIpOnLaunch = True
tags = {
"kubernetes.io/role/elb": "1"
}
}
}
} for _zone in _zones]
_rpas = [
{name = "controlplane", role = "controlplane", policyArn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"}
{name = "service", role = "controlplane", policyArn = "arn:aws:iam::aws:policy/AmazonEKSServicePolicy"}
{name = "worker", role = "nodegroup", policyArn = "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"}
{name = "cni", role = "nodegroup", policyArn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"}
{name = "registry", role = "nodegroup", policyArn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"}
]
_items += [{
apiVersion = "iam.aws.upbound.io/v1beta1"
kind = "RolePolicyAttachment"
metadata = _metadata(_rpa.name, "iamattachment-" + _rpa.name, "")
spec = {
forProvider = {
policyArn = _rpa.policyArn
roleSelector = {
matchControllerRef = True
matchLabels.role = oxr.spec.id + "-" + _rpa.role
}
}
}
} for _rpa in _rpas]
_roles = [
{name = "controlplane", service = "eks"}
{name = "nodegroup", service = "ec2"}
]
_items += [{
apiVersion = "iam.aws.upbound.io/v1beta1"
kind = "Role"
metadata = {
name = oxr.spec.id + "-" + _role.name
annotations = {
"krm.kcl.dev/composition-resource-name" = "iamrole-" + _role.name
}
labels.role = oxr.spec.id + "-" + _role.name
}
spec.forProvider.assumeRolePolicy = """\
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {"Service": [\"""" + _role.service + """.amazonaws.com"]},
"Action": ["sts:AssumeRole"]
}]
}
"""
} for _role in _roles]
items = _items
- step: apps
functionRef:
name: crossplane-contrib-function-kcl
input:
apiVersion: krm.kcl.dev/v1alpha1
kind: KCLRun
spec:
source: |
crossplane = "1.14.5"
argocd = "3.35.4"
dapr = "1.12.4"
traefik = "26.0.0"
dynatraceOperator = "0.15.0"
dynatraceDashboard = "0.2.2"
externalSecrets = "0.9.11"
cilium = "1.14.2"
openFunctionUrl = "https://openfunction.github.io/charts/openfunction-v1.2.0-v0.7.0.tgz"
oxr = option("params").oxr
ocds = option("params").ocds
_metadata = lambda suffix: str -> any {
{
name = oxr.spec.id
annotations = {
"crossplane.io/external-name" = oxr.spec.id
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-" + suffix
}
}
}
schema usage:
_nameSuffix: str
_kind: str = "Object"
apiVersion = "apiextensions.crossplane.io/v1alpha1"
kind = "Usage"
metadata = {
name = oxr.spec.id + "-" + _nameSuffix + "-usage"
annotations = {
"crossplane.io/external-name" = oxr.spec.id + "-" + _nameSuffix + "-usage"
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-" + _nameSuffix + "-usage"
}
}
spec = {
of = {
apiVersion = "eks.aws.upbound.io/v1beta1"
kind = "Cluster"
resourceRef.name = oxr.spec.id
}
by = {
if _kind == "Object":
apiVersion = "kubernetes.crossplane.io/v1alpha2"
elif _kind == "Release":
apiVersion = "helm.crossplane.io/v1beta1"
kind = _kind
resourceRef.name = oxr.spec.id + "-" + _nameSuffix
}
}
schema chart:
_name: str
_chartName?: str
_chartRepository?: str
_chartVersion?: str
_chartUrl?: str
_namespace: str
_values?: any
_providerConfigRefName?: str
apiVersion = "helm.crossplane.io/v1beta1"
kind = "Release"
metadata = {
name = oxr.spec.id + "-app-" + _name
annotations = {
"crossplane.io/external-name" = _name
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-app-" + _name
}
}
spec = {
forProvider = {
chart = {
if _chartName:
name = _chartName
else:
name = _name
if _chartRepository:
repository = _chartRepository
if _chartVersion:
version = _chartVersion
if _chartUrl:
url = _chartUrl
}
if _values:
values = _values
namespace = _namespace
}
rollbackLimit = 3
if _providerConfigRefName:
providerConfigRef.name = _providerConfigRefName
else:
providerConfigRef.name = oxr.spec.id
}
schema object:
_name: str
_externalName?: str
_manifest: any
_references?: []
apiVersion = "kubernetes.crossplane.io/v1alpha2"
kind = "Object"
metadata = {
name = oxr.spec.id + "-app-" + _name
annotations = {
if _externalName:
"crossplane.io/external-name" = _externalName
else:
"crossplane.io/external-name" = _name
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-app-" + _name
}
}
spec = {
if _references:
references = _references
forProvider.manifest = _manifest
providerConfigRef.name = oxr.spec.id
}
_items = [
{
apiVersion = "helm.crossplane.io/v1beta1"
kind = "ProviderConfig"
metadata = {
name = oxr.spec.id
annotations = {
"krm.kcl.dev/ready": "True"
"crossplane.io/external-name" = oxr.spec.id
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-config-helm"
}
}
spec = {
credentials = {
secretRef = {
namespace = oxr.spec.claimRef.namespace
key = "kubeconfig"
name = oxr.spec.id + "-cluster"
}
source = "Secret"
}
if oxr.spec.compositionSelector.matchLabels.provider == "google":
identity = {
type = "GoogleApplicationCredentials"
source = "Secret"
secretRef = {
name = "gcp-creds"
namespace = "crossplane-system"
key = "creds"
}
}
}
}
{
apiVersion = "helm.crossplane.io/v1beta1"
kind = "ProviderConfig"
metadata = {
name = oxr.spec.id + "-local"
annotations = {
"krm.kcl.dev/ready": "True"
"crossplane.io/external-name" = oxr.spec.id
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-config-helm-local"
}
}
spec.credentials.source = "InjectedIdentity"
}
if oxr.spec.compositionSelector.matchLabels.provider != "aws":
{
apiVersion = "helm.crossplane.io/v1beta1"
kind = "Release"
metadata = {
name = oxr.spec.id + "-cilium"
annotations = {
"crossplane.io/external-name" = "cilium"
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-app-cilium"
}
}
spec = {
forProvider = {
chart = {
name = "cilium"
repository = "https://helm.cilium.io"
version = cilium
}
set = [
if oxr.spec.compositionSelector.matchLabels.provider == "google":
{name = "nodeinit.enabled", value = "true"}
{name = "nodeinit.reconfigureKubelet", value = "true"}
{name = "nodeinit.removeCbrBridge", value = "true"}
{name = "cni.binPath", value = "/home/kubernetes/bin"}
{name = "gke.enabled", value = "true"}
{name = "ipam.mode", value = "kubernetes"}
{
name = "ipv4NativeRoutingCIDR"
if oxr.spec.id + "-nodepool" in ocds:
value = oxr.status.field1
},
{name = "authentication.mutual.spire.enabled", value = "true"}
{name = "authentication.mutual.spire.install.enabled", value = "true"}
if oxr.spec.compositionSelector.matchLabels.provider == "azure":
{name = "aksbyocni.enabled", value = "true"}
{name = "nodeinit.enabled", value = "true"}
{name = "authentication.mutual.spire.enabled", value = "true"}
{name = "authentication.mutual.spire.install.enabled", value = "true"}
]
namespace = "kube-system"
}
rollbackLimit = 3
providerConfigRef.name = oxr.spec.id
}
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "cilium", _kind = "Release" }
{
apiVersion = "kubernetes.crossplane.io/v1alpha1"
kind = "ProviderConfig"
metadata = {
name = oxr.spec.id
annotations = {
"krm.kcl.dev/ready": "True"
"crossplane.io/external-name" = oxr.spec.id
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-config-kubernetes"
}
}
spec = {
credentials = {
secretRef = {
key = "kubeconfig"
name = oxr.spec.id + "-cluster"
namespace = oxr.spec.claimRef.namespace
}
source: "Secret"
}
if oxr.spec.compositionSelector.matchLabels.provider == "google":
identity = {
type = "GoogleApplicationCredentials"
source = "Secret"
secretRef = {
name = "gcp-creds"
namespace = "crossplane-system"
key = "creds"
}
}
}
}
if oxr.spec.parameters?.apps?.crossplane?.enabled:
chart {
_name = "crossplane"
_chartRepository = "https://charts.crossplane.io/stable"
_chartVersion = crossplane
_namespace = "crossplane-system"
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "crossplane", _kind = "Release" }
if oxr.spec.parameters?.apps?.argocd?.enabled:
chart {
_name = "argo-cd"
_chartRepository = "https://argoproj.github.io/argo-helm"
_chartVersion = argocd
_namespace = "argocd"
_values = {
global.domain = oxr.spec.parameters.apps.argocd.host
configs = {
secret = {
argocdServerAdminPassword = "$2a$10$m3eTlEdRen0nS86c5Zph5u/bDFQMcWZYdG3NVdiyaACCqoxLJaz16"
argocdServerAdminPasswordMtime = "2021-11-08T15:04:05Z"
}
cm = {
"application.resourceTrackingMethod" = "annotation"
"timeout.reconciliation" = "60s"
}
params = {
"server.insecure" = True
}
}
server = {
if oxr.spec.parameters?.apps?.traefik?.enabled:
ingress = {
enabled = True
ingressClassName = "traefik"
}
extraArgs = ["--insecure"]
}
}
}
object {
_name = "argo-cd-app"
_manifest = {
apiVersion = "argoproj.io/v1alpha1"
kind = "Application"
metadata = {
name = "apps"
namespace = "argocd"
finalizers = ["resources-finalizer.argocd.argoproj.io"]
}
spec = {
project = "default"
source = {
repoURL = oxr.spec.parameters.apps.argocd.repoURL
targetRevision = "HEAD"
path = oxr.spec.parameters.apps.argocd.sourcePath
}
destination = {
server = "https://kubernetes.default.svc"
namespace = oxr.spec.parameters.apps.argocd.destinationNamespace
}
syncPolicy.automated = {
selfHeal = True
prune = True
allowEmpty = True
}
}
}
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "argo-cd", _kind = "Release" }
usage { _nameSuffix = "argo-cd-app" }
if oxr.spec.parameters?.apps?.openfunction?.enabled:
chart {
_name = "openfunction"
_chartUrl = openFunctionUrl
_namespace = "openfunction"
_values = {
revisionController.enable = True
}
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "openfunction", _kind = "Release" }
if oxr.spec.parameters?.apps?.dapr?.enabled:
chart {
_name = "dapr"
_chartRepository = "https://dapr.github.io/helm-charts/"
_chartVersion = dapr
_namespace = "dapr-system"
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "dapr", _kind = "Release" }
if oxr.spec.parameters?.apps?.traefik?.enabled:
chart {
_name = "traefik"
_chartRepository = "https://helm.traefik.io/traefik"
_chartVersion = traefik
_namespace = "traefik"
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "traefik", _kind = "Release" }
if oxr.spec.parameters?.apps?.dynatrace?.enabled:
chart {
_name = "dynatrace-operator"
_chartRepository = "https://raw.githubusercontent.com/Dynatrace/dynatrace-operator/main/config/helm/repos/stable"
_chartVersion = dynatraceOperator
_namespace = "dynatrace"
_values = {
installCRD = True
csidriver.enabled = True
}
}
object {
_name = "dynakube"
_manifest = {
apiVersion = "dynatrace.com/v1beta1"
kind = "DynaKube"
metadata = {
name = oxr.spec.id
namespace = "dynatrace"
annotations = {
"feature.dynatrace.com/k8s-app-enabled" = "true"
}
}
spec = {
apiUrl = oxr.spec.parameters.apps.dynatrace.apiUrl
oneAgent.cloudNativeFullStack.image = ""
activeGate = {
capabilities = [
"kubernetes-monitoring"
"routing"
"metrics-ingest"
"dynatrace-api"
]
image = ""
resources = {
requests = {
cpu = "500m"
memory = "512Mi"
}
limits = {
cpu = "1000m"
memory = "1.5Gi"
}
}
}
}
}
}
chart {
_name = "dynatrace-dashboard"
_chartName = "kubernetes-cluster"
_chartRepository = "https://katharinasick.github.io/crossplane-observability-demo-dynatrace"
_chartVersion = dynatraceDashboard
_namespace = "dynatrace"
_values = {
oauthCredentialsSecretName = oxr.spec.parameters.apps.dynatrace.oathCredentialsSecretName
cluster = oxr.spec.id
dashboards = {
clusterOverview.enabled = True
crossplaneMetrics.enabled = False
}
}
_providerConfigRefName = oxr.spec.id + "-local"
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "dynatrace-operator", _kind = "Release" }
usage { _nameSuffix = "dynakube" }
usage { _nameSuffix = "dynatrace-dashboard", _kind = "Release" }
if oxr.spec.parameters?.apps?.externalSecrets?.enabled:
chart {
_name = "external-secrets"
_chartRepository = "https://charts.external-secrets.io"
_chartVersion = externalSecrets
_namespace = "external-secrets"
_values = {
installCRDs = True
}
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "external-secrets", _kind = "Release" }
if oxr.spec.parameters?.apps?.externalSecrets?.enabled and oxr.spec.parameters.apps?.externalSecrets?.store:
object {
_name = "secret-store"
_externalName = oxr.spec.compositionSelector.matchLabels.provider
_manifest = {
apiVersion = "external-secrets.io/v1beta1"
kind = "ClusterSecretStore"
metadata.name = oxr.spec.compositionSelector.matchLabels.provider
if oxr.spec.compositionSelector.matchLabels.provider == "google":
spec.provider.gcpsm.auth.secretRef.secretAccessKeySecretRef = {
name: "gcp-creds"
key: oxr.spec.parameters.apps.externalSecrets.googleCredentialsKey
namespace: oxr.spec.parameters.creds.namespace
}
elif oxr.spec.compositionSelector.matchLabels.provider == "azure":
spec.provider.azurekv = {
authType = "ManagedIdentity"
vaultUrl = oxr.spec.parameters.apps.externalSecrets.azureVaultUrl
}
elif oxr.spec.compositionSelector.matchLabels.provider == "aws":
spec.provider.aws = {
service = "SecretsManager"
region = "us-east-1"
auth.secretRef = {
accessKeyIDSecretRef = {
name = oxr.spec.parameters.creds.name
key = oxr.spec.parameters.apps.externalSecrets.awsAccessKeyIDKey
namespace = oxr.spec.parameters.creds.namespace
}
secretAccessKeySecretRef = {
name = oxr.spec.parameters.creds.name
key = oxr.spec.parameters.apps.externalSecrets.awsSecretAccessKeyKey
namespace = oxr.spec.parameters.creds.namespace
}
}
}
}
if oxr.spec.compositionSelector.matchLabels.provider == "google":
_references = [{
patchesFrom = {
apiVersion = "gcp.upbound.io/v1beta1"
kind = "ProviderConfig"
name = "default"
fieldPath = "spec.projectID"
}
toFieldPath = "spec.provider.gcpsm.projectID"
}]
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "secret-store" }
]
if oxr.spec.parameters?.apps?.externalSecrets?.secrets:
_items += [{
apiVersion = "kubernetes.crossplane.io/v1alpha2"
kind = "Object"
metadata = {
name = oxr.spec.id + "-secret-" + _secret.toSecret
annotations = {
"crossplane.io/external-name" = _secret.toSecret
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-secret-" + _secret.toSecret
}
}
spec = {
forProvider.manifest = {
apiVersion = "external-secrets.io/v1beta1"
kind = "ExternalSecret"
metadata = {
name: _secret.toSecret
namespace: _secret.toNamespace
}
spec = {
refreshInterval = "1h"
secretStoreRef = {
kind: "ClusterSecretStore"
name: oxr.spec.compositionSelector.matchLabels.provider
}
target = {
name = _secret.toSecret
creationPolicy = "Owner"
template.type = _secret.type
}
dataFrom = [{ extract.key = _secret.fromSecret }]
}
}
providerConfigRef.name = oxr.spec.id
}
} for _secret in oxr.spec.parameters.apps.externalSecrets.secrets ]
if oxr.spec.parameters?.usage?.enabled:
_items += [ usage {
_nameSuffix = "secret-" + _secret.toSecret
} for _secret in oxr.spec.parameters.apps.externalSecrets.secrets ]
if oxr.spec.parameters?.namespaces:
_items += [{
apiVersion = "kubernetes.crossplane.io/v1alpha2"
kind = "Object"
metadata = {
name = oxr.spec.id + "-ns-" + _namespace
annotations = {
"crossplane.io/external-name" = _namespace
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-ns-" + _namespace
}
}
spec = {
forProvider.manifest = {
apiVersion = "v1"
kind = "Namespace"
metadata.name = _namespace
}
deletionPolicy = "Orphan"
providerConfigRef.name = oxr.spec.id
}
} for _namespace in oxr.spec.parameters.namespaces ]
if oxr.spec.parameters?.creds:
_items += [
{
apiVersion = "kubernetes.crossplane.io/v1alpha2"
kind = "Object"
metadata = {
name = oxr.spec.id + "-creds"
annotations = {
"crossplane.io/external-name" = oxr.spec.parameters.creds.name
"krm.kcl.dev/composition-resource-name" = oxr.spec.id + "-creds"
}
}
spec = {
references = [{
patchesFrom = {
apiVersion = "v1"
kind = "Secret"
name: oxr.spec.parameters.creds.name
namespace: oxr.spec.parameters.creds.namespace
fieldPath: "data." + _credReference
}
toFieldPath: "data." + _credReference
} for _credReference in oxr.spec.parameters.creds.keys]
forProvider = {
manifest = {
apiVersion = "v1"
kind = "Secret"
metadata = {
name = oxr.spec.parameters.creds.name
namespace = oxr.spec.parameters.creds.namespace
}
}
}
providerConfigRef.name = oxr.spec.id
}
}
if oxr.spec.parameters?.usage?.enabled:
usage { _nameSuffix = "creds" }
]
items = _items
- step: automatically-detect-ready-composed-resources
functionRef:
name: crossplane-contrib-function-auto-ready
writeConnectionSecretsToNamespace: crossplane-system
If you change the KCL version in https://github.com/vfarcic/crossplane-kubernetes/blob/main/providers/function-kcl.yaml, you can execute the following to reproduce it:
A shell with all the tools as Nix packages. Devbox can be installed from https://www.jetify.com/devbox. This step is optional if you already have
justas a few other tools.
devbox shellCreate a cluster with everything in it.
just cluster-createRun the test watcher. It will fail if a newer version of KCL is in https://github.com/vfarcic/crossplane-kubernetes/blob/main/providers/function-kcl.yaml. The same tests work if KCL version 0.9.0 is used.
just test-watchThe block of code that produces the issue is in https://github.com/vfarcic/crossplane-kubernetes/blob/main/kcl/apps.k#L531. There might be others but the tests stop when the first issue is found so I haven't verified what comes afterward.
Stop the watcher with
ctrl+cand execute the command that follows to destroy everything.
just cluster-destroyIs that what you're looking for? Please let me know if it isn't (and I probably misunderstood your message).
Peefy
commented
2 months ago Thank you! @vfarcic
Just one XR and Composition YAML are enough. May I ask if I can add it directly to the e2e test of function-kcl repo for testing before release?
vfarcic
commented
2 months ago Sorry... I misunderstood your initial question.
Feel free to use it in any form or way you think might help the project.
Peefy
commented
2 months ago No worries.
I have reproduced the issues which may have been caused by an internal error in KCL. I have released crossplane function-kcl v0.9.4 to roll back the feature of duplicate resource checking and align it with v0.9.0.
vfarcic
commented
2 months ago v0.9.4 works :) Thanks a ton.
Since v0.9.2 I'm getting the following error (event in the composition):
The snippet of the Composition that produces that error is:
I am certain that
oxr.spec.parameters.namespacescontains unique values (no duplication). More importantly, exactly the same code worked with v0.9.0. The only change I did was to upgrade the function to v0.9.2.