eks: Reconcile error with SubnetIDs

[jeunii]

What happened?

I am creating a composition that creates the following managed resources

• vpc
• 2 subnets ( 1 public and 1 private)
• Internet GW
• routetable
• sg
• role
• EKS cluster

All resources except for the cluster are being created. The error I am getting is

k describe cluster.eks.aws.upbound.io my-eks-setup-qb82m
Name:         my-eks-setup-qb82m
...
Status:
  At Provider:
  Conditions:
    Last Transition Time:  2022-11-03T15:58:55Z
    Message:               observe failed: cannot run refresh: refresh failed: Missing required argument: The argument "subnet_ids" is required, but no definition was found.
Too many vpc_config blocks: No more than 1 "vpc_config" blocks are allowed
    Reason:  ReconcileError
    Status:  False
    Type:    Synced
Events:
  Type     Reason                         Age                   From                                              Message
  ----     ------                         ----                  ----                                              -------
  Warning  CannotObserveExternalResource  4m28s (x37 over 29m)  managed/eks.aws.upbound.io/v1beta1, kind=cluster  cannot run refresh: refresh failed: Missing required argument: The argument "subnet_ids" is required, but no definition was found.
Too many vpc_config blocks: No more than 1 "vpc_config" blocks are allowed

Although the subnet is created and has been picked up the cluster managed resource

k describe cluster.eks.aws.upbound.io my-eks-setup-qb82m
Name:         my-eks-setup-qb82m
...
Spec:
  Deletion Policy:  Delete
  For Provider:
    Region:    ca-central-1
    Role Arn:  arn:aws:iam::744734775600:role/my-eks-setup-zrpws. <----- Correctly populated
    Role Arn Ref:
      Name:  my-eks-setup-zrpws
    Role Arn Selector:
      Match Controller Ref:  true
      Match Labels:
        Role:  controlplane
    Tags:
      Crossplane - Kind:            cluster.eks.aws.upbound.io
      Crossplane - Name:            my-eks-setup-qb82m
      Crossplane - Providerconfig:  default
    Version:                        1.23
    Vpc Config:
      Subnet Id Refs:
        Name:  my-eks-setup-g2z6r
      Subnet Id Selector:
        Match Controller Ref:  true
        Match Labels:
          Type:  public
      Subnet Ids:
        subnet-068ecf210baf00f0f  !!!!!!!!!!!!!!!!!!!!  <----- SUBNET Correctly populated !!!!!!!!!!!!!!!!!!!!
      Security Group Id Refs:
        Name:  my-eks-setup-sh97m
      Security Group Id Selector:
        Match Controller Ref:  true
        Match Labels:
          For:  cluster
      Security Group Ids:
        sg-08be552961b0bbd66.   <----- Correctly populated 
  Provider Config Ref:
    Name:  default

Everything else seems to be correctly deployed

➜ k get vpc
NAME                 READY   SYNCED   EXTERNAL-NAME           AGE
my-eks-setup-lnrvb   True    True     vpc-07432f2383803c75b   83m

❯ k get subnets
NAME                 READY   SYNCED   EXTERNAL-NAME              AGE
my-eks-setup-bjzjr   True    True     subnet-0f75991a82d2839b7   81m
my-eks-setup-g2z6r   True    True     subnet-068ecf210baf00f0f   81m

➜ k get securitygroups
NAME                 READY   SYNCED   EXTERNAL-NAME          AGE
my-eks-setup-sh97m   True    True     sg-08be552961b0bbd66   83m

➜ k get roles.iam.aws.upbound.io
NAME                 READY   SYNCED   EXTERNAL-NAME                                    AGE
my-eks-setup-zrpws   True    True     my-eks-setup-zrpws                               83m

How can we reproduce it?

Here are my configs

XRD

apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: xdemoekss.eks.example.org
spec:
  group: eks.example.org
  names:
    kind: XDemoEKS
    plural: xdemoekss
  claimNames:
    kind: DemoEKS
    plural: demoekss
  connectionSecretKeys:
  - kubeconfig
  versions:
  - name: v1alpha1
    served: true
    referenceable: true
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            properties:
              parameters:
                type: object
                properties:
                  kubeVersion:
                    type: string
                    description: The kubernetes control plane version.
                required:
                  - kubeVersion
            required:
              - parameters

Composition

---
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: demoeks.eks.example.org
  labels:
    provider: aws
spec:
  compositeTypeRef:
    apiVersion: eks.example.org/v1alpha1
    kind: XDemoEKS
  resources:
    - name: vpc
      base:
        apiVersion: ec2.aws.upbound.io/v1beta1
        kind: VPC
        spec:
          forProvider:
            region: ca-central-1
            cidrBlock: 192.168.0.0/16
            enableDnsSupport: true
            enableDnsHostNames: true
            tags:
              Name: demo-vpc
    - name: subnet-a
      base:
        apiVersion: ec2.aws.upbound.io/v1beta1
        kind: Subnet
        metadata:
          labels:
            zone: ca-central-1a
            type: public
        spec:
          forProvider:
            region: ca-central-1
            cidrBlock: 192.168.64.0/18
            vpcIdSelector:
              matchControllerRef: true
            availabilityZone: ca-central-1a
    - name: subnet-b
      base:
        apiVersion: ec2.aws.upbound.io/v1beta1
        kind: Subnet
        metadata:
          labels:
            zone: ca-central-1b
            type: private
        spec:
          forProvider:
            region: ca-central-1
            cidrBlock: 192.168.128.0/18
            vpcIdSelector:
              matchControllerRef: true
            availabilityZone: ca-central-1b
    - name: internetgateway
      base:
        apiVersion: ec2.aws.upbound.io/v1beta1
        kind: InternetGateway
        spec:
          forProvider:
            region: ca-central-1
            vpcIdSelector:
              matchControllerRef: true
    - name: routetable
      base:
        apiVersion: ec2.aws.upbound.io/v1beta1
        kind: RouteTable
        spec:
          forProvider:
            region: ca-central-1
            vpcIdSelector:
              matchControllerRef: true
            routes:
              - destinationCidrBlock: 0.0.0.0/0
                gatewayIdSelector:
                  matchControllerRef: true
            associations:
              - subnetIdSelector:
                  matchLabels:
                    zone: ca-central-1a
              - subnetIdSelector:
                  matchLabels:
                    zone: ca-central-1b
              - subnetIdSelector:
                  matchLabels:
                    zone: ca-central-1d
    - name: eks-cluster-sg
      base:
        apiVersion: ec2.aws.upbound.io/v1beta1
        kind: SecurityGroup
        metadata:
          name: sg
          labels:
            for: cluster
        spec:
          forProvider:
            region: ca-central-1
            tags:
              type: upbound-test-security-group
            vpcIdSelector:
              matchControllerRef: true
    - name: eks-cluster-sg-rule
      base:
        apiVersion: ec2.aws.upbound.io/v1beta1
        kind: SecurityGroupRule
        metadata:
          name: example
          labels:
            for: cluster
        spec:
          forProvider:
            cidrBlocks:
              - 10.0.0.0/16
            fromPort: 0
            protocol: tcp
            region: ca-central-1
            securityGroupIdSelector:
              matchControllerRef: true
            toPort: 65535
            type: ingress
    - name: controlplaneRole
      base:
        apiVersion: iam.aws.upbound.io/v1beta1
        kind: Role
        metadata:
          labels:
            role: controlplane
        spec:
          forProvider:
            assumeRolePolicy: |
              {
                "Version": "2012-10-17",
                "Statement": [
                    {
                        "Effect": "Allow",
                        "Principal": {
                            "Service": [
                                "eks.amazonaws.com"
                            ]
                        },
                        "Action": [
                            "sts:AssumeRole"
                        ]
                    }
                ]
              }
    - name: clusterRolePolicyAttachment
      base:
        apiVersion: iam.aws.upbound.io/v1beta1
        kind: RolePolicyAttachment
        spec:
          forProvider:
            policyArn: arn:aws:iam::aws:policy/AmazonEKSClusterPolicy
            roleSelector:
              matchControllerRef: true
              matchLabels:
                role: controlplane
    - name: eks-cluster
      base:
        apiVersion: eks.aws.upbound.io/v1beta1
        kind: Cluster
        metadata:
          name: demo-eks-cluster-role
        spec:
          forProvider:
            region: ca-central-1
            roleArnSelector:
              matchControllerRef: true
              matchLabels:
                role: controlplane
            vpcConfig:
            - subnetIdSelector:
                matchControllerRef: true
                matchLabels:
                  type: public
            - securityGroupIdSelector:
                matchControllerRef: true
                matchLabels:
                  for: cluster
      patches:
        - fromFieldPath: spec.parameters.kubeVersion
          toFieldPath: spec.forProvider.version

XR

apiVersion: eks.example.org/v1alpha1
kind: XDemoEKS
metadata:
  name: my-eks-setup
spec:
  parameters:
    kubeVersion: "1.23"
  compositionRef:
    name: demoeks.eks.example.org

What environment did it happen in?

• Universal Crossplane Version: v1.10.0
• Provider Version: v1.18.0

[fcastarede]

Same issue with:

• Universal Crossplane Version: v1.10.1-up.1
• Provider version: v0.21.0

[geowalrus4gh]

still an issue with provider latest v0.37.0 ? I was trying to migrate from community provider to official Upbound provider. Am not able to create EKS resources. Getting this error when testing EKS creation.

 Warning  CannotObserveExternalResource  15s (x5 over 27s)  managed/eks.aws.upbound.io/v1beta1, kind=cluster  cannot run refresh: refresh failed: Missing required argument: The argument "subnet_ids" is required, but no definition was found.
Missing required argument: The argument "subnet_ids" is required, but no definition was found.
Too many vpc_config blocks: No more than 1 "vpc_config" blocks are allowed

Any pointers to this please ?

[geowalrus4gh]

Provider renders the Cluster spec to terraform config as snippet below. "vpc_config":[ { "subnet_ids":[ "subnet-xxx", "subnet-xxx" ] }, { "public_access_cidrs":[ "0.0.0.0/0" ] }, { "security_group_ids":[ "sg-xx" ] }, { "endpoint_private_access":true }, { "endpoint_public_access":false } ]

In terraform vpc_config is not an array. It's an object. Like below

"vpc_config":{ "subnet_ids":[ "subnet-xx", "subnet-xx" ], "public_access_cidrs":[ "0.0.0.0/0" ], "security_group_ids":[ "sg-xx" ], "endpoint_private_access":true, "endpoint_public_access":false }

[geowalrus4gh]

@jeanduplessis any workaround for this available?