crossplane / upjet

A code generation framework and runtime for Crossplane providers
Apache License 2.0
293 stars 84 forks source link

Cannot import existing resources without providing the exact required field values #2

Closed ezgidemirel closed 1 year ago

ezgidemirel commented 2 years ago

What happened?

Tried to import an existing subnet resource in provider-jet-aws with following manifest:

apiVersion: ec2.aws.jet.crossplane.io/v1alpha2
kind: Subnet
metadata:
  annotations:
    crossplane.io/external-name: subnet-03c0119e7ed0b4847
  name: ezgi-subnet
spec:
  forProvider:
    region: us-west-1
    cidrBlock: ""
    vpcId: ""

cidrBlock and vpcId fields are required and need to be provided for schema validation. However, passing empty values to those resources causes the following error:

Warning  CannotObserveExternalResource  19s (x5 over 31s)  managed/ec2.aws.jet.crossplane.io/v1alpha2, kind=subnet  cannot run refresh: refresh failed: Missing required argument: The argument "vpc_id" is required, but no definition was found.: File name: main.tf.json

Importing a subnet resource is possible by Terraform with only external-name info (please see here for more information). Therefore, the manifest above should work.

This use case is especially important for customers who will migrate from native providers to jet providers.

How can we reproduce it?

Create a subnet resource with the following manifest:

apiVersion: ec2.aws.jet.crossplane.io/v1alpha2
kind: Subnet
metadata:
  name: ezgi-subnet
spec:
  deletePolicy: Orphan
  forProvider:
    region: us-west-1
    availabilityZone: us-west-1b
    vpcIdRef:
      name: ezgi-vpc
    cidrBlock: 172.16.10.0/24

Delete the MR and try to import it with the first manifest.

mcanevet commented 1 year ago

Hello, I'd really like to be able to import an existing resource without having to fill all required fields. This is possible in other IaC tools like Terraform Datasource of ACK Adopted resources where only the fields that allows to retrieve the resource in an unambiguous way are required. The use case would be to use attributes of a resource that is note managed by Crossplane (could be legacy stuffs managed manually or by another IaC tool, or even someone else's Crossplane) in our resources.

For example, we have an EKS platform currently deployed using Terraform where our users are deploying all their applications. Unfortunately, when their application needs IAM authentication, there are currently creating an IAM role with an assume role policy that references the OIDC issuer URL of the EKS cluster that is known by Terraform because it manages the EKS resource. We'd like to remove this adhesion between the platform and the workload and allow our users to create their IAM roles through Kubernetes resources (juste like their Deployments). But for that we need to retrieve the OIDC issuer URL of the current cluster dynamically.

turkenh commented 1 year ago

This is also relevant to Observe Only feature, which is currently in the design phase.

As part of that work, I believe we will need to enhance upjet to use terrraform import instead of terraform apply --refresh-only to fetch the resource state from external APIs. I hope this will fix this issue as well.

haarchri commented 1 year ago

ObserveOnly is implemented in official providers as Alpha feature - so we can close this issue