search

crossplane / upjet

A code generation framework and runtime for Crossplane providers
Apache License 2.0
296 stars 86 forks source link

Code generation error: Import cycle not allowed #368

Open mbbush opened 6 months ago

mbbush commented 6 months ago

What happened?

I've noticed that since the conversion webhooks were added, when making certain otherwise-valid changes to the resource configuration in provider-aws, the code generation pipeline returns errors. And even when it succeeds, it emits warnings about import cycles.

Warnings on successful codegen:

17:15:47 [ .. ] verify go modules dependencies have expected content
all modules verified
17:15:50 [ OK ] go modules dependencies verified
17:15:50 [ .. ] generating provider schema for hashicorp/aws 5.31.0
17:15:55 [ OK ] generating provider schema for hashicorp/aws 5.31.0

...

17:15:58 [ .. ] go generate linux_amd64

Generated 933 resources!
{"level":"info","ts":"2024-03-08T17:22:12-08:00","logger":"transformer-resolver","msg":"Encounter the following issues when loading a package","package":"v1beta1","issues":"-: import cycle not allowed: import stack: [github.com/upbound/provider-aws/apis github.com/upbound/provider-aws/apis/autoscaling/v1beta1 github.com/upbound/provider-aws/apis/autoscaling/v1beta2 github.com/upbound/provider-aws/apis/autoscaling/v1beta1]\n/home/matt/git/upbound-provider-aws/apis/autoscaling/v1beta1/zz_generated.resolvers.go:13:10: could not import github.com/upbound/provider-aws/apis/autoscaling/v1beta2 (import cycle: [github.com/upbound/provider-aws/apis/autoscaling/v1beta2 github.com/upbound/provider-aws/apis/autoscaling/v1beta1])\n"}
{"level":"info","ts":"2024-03-08T17:22:12-08:00","logger":"transformer-resolver","msg":"Encounter the following issues when loading a package","package":"v1beta1","issues":"-: import cycle not allowed: import stack: [github.com/upbound/provider-aws/apis github.com/upbound/provider-aws/apis/connect/v1beta1 github.com/upbound/provider-aws/apis/connect/v1beta2 github.com/upbound/provider-aws/apis/connect/v1beta1]\n/home/matt/git/upbound-provider-aws/apis/connect/v1beta1/zz_generated.resolvers.go:13:10: could not import github.com/upbound/provider-aws/apis/connect/v1beta2 (import cycle: [github.com/upbound/provider-aws/apis/connect/v1beta2 github.com/upbound/provider-aws/apis/connect/v1beta1])\n"}
{"level":"info","ts":"2024-03-08T17:22:12-08:00","logger":"transformer-resolver","msg":"Encounter the following issues when loading a package","package":"v1beta1","issues":"-: import cycle not allowed: import stack: [github.com/upbound/provider-aws/apis github.com/upbound/provider-aws/apis/elasticache/v1beta1 github.com/upbound/provider-aws/apis/elasticache/v1beta2 github.com/upbound/provider-aws/apis/elasticache/v1beta1]\n/home/matt/git/upbound-provider-aws/apis/elasticache/v1beta1/zz_generated.resolvers.go:14:10: could not import github.com/upbound/provider-aws/apis/elasticache/v1beta2 (import cycle: [github.com/upbound/provider-aws/apis/elasticache/v1beta2 github.com/upbound/provider-aws/apis/elasticache/v1beta1])\n"}
{"level":"info","ts":"2024-03-08T17:22:12-08:00","logger":"transformer-resolver","msg":"Encounter the following issues when loading a package","package":"v1beta1","issues":"-: import cycle not allowed: import stack: [github.com/upbound/provider-aws/apis github.com/upbound/provider-aws/apis/kafka/v1beta1 github.com/upbound/provider-aws/apis/kafka/v1beta2 github.com/upbound/provider-aws/apis/kafka/v1beta1]\n/home/matt/git/upbound-provider-aws/apis/kafka/v1beta1/zz_generated.resolvers.go:16:10: could not import github.com/upbound/provider-aws/apis/kafka/v1beta2 (import cycle: [github.com/upbound/provider-aws/apis/kafka/v1beta2 github.com/upbound/provider-aws/apis/kafka/v1beta1])\n"}
{"level":"info","ts":"2024-03-08T17:22:12-08:00","logger":"transformer-resolver","msg":"Encounter the following issues when loading a package","package":"v1beta2","issues":"-: import cycle not allowed: import stack: [github.com/upbound/provider-aws/apis github.com/upbound/provider-aws/apis/opsworks/v1beta1 github.com/upbound/provider-aws/apis/rds/v1beta2 github.com/upbound/provider-aws/apis/rds/v1beta1 github.com/upbound/provider-aws/apis/rds/v1beta2]\n/home/matt/git/upbound-provider-aws/apis/rds/v1beta2/zz_generated.resolvers.go:16:10: could not import github.com/upbound/provider-aws/apis/rds/v1beta1 (import cycle: [github.com/upbound/provider-aws/apis/rds/v1beta1 github.com/upbound/provider-aws/apis/rds/v1beta2])\n"}
17:22:13 [ OK ] go generate linux_amd64
17:22:13 [ .. ] go mod tidy
17:22:13 [ OK ] go mod tidy

Errors during unsuccessful codegen:

18:17:58 [ .. ] verify go modules dependencies have expected content
all modules verified
18:18:00 [ OK ] go modules dependencies verified
18:18:00 [ .. ] generating provider schema for hashicorp/aws 5.31.0
18:18:02 [ OK ] generating provider schema for hashicorp/aws 5.31.0

...

18:18:05 [ .. ] go generate linux_amd64
package command-line-arguments
    imports github.com/upbound/provider-aws/config
    imports github.com/upbound/provider-aws/config/autoscaling
    imports github.com/upbound/provider-aws/apis/autoscaling/v1beta1
    imports github.com/upbound/provider-aws/apis/autoscaling/v1beta2
    imports github.com/upbound/provider-aws/apis/autoscaling/v1beta1: import cycle not allowed
package command-line-arguments
    imports github.com/upbound/provider-aws/config
    imports github.com/upbound/provider-aws/config/connect
    imports github.com/upbound/provider-aws/apis/connect/v1beta1
    imports github.com/upbound/provider-aws/apis/connect/v1beta2
    imports github.com/upbound/provider-aws/apis/connect/v1beta1: import cycle not allowed
package command-line-arguments
    imports github.com/upbound/provider-aws/config
    imports github.com/upbound/provider-aws/config/elasticache
    imports github.com/upbound/provider-aws/apis/elasticache/v1beta1
    imports github.com/upbound/provider-aws/apis/elasticache/v1beta2
    imports github.com/upbound/provider-aws/apis/elasticache/v1beta1: import cycle not allowed
apis/generate.go:28: running "go": exit status 1
18:18:06 [FAIL]
make[1]: *** [build/makelib/golang.mk:240: go.generate] Error 1
make: *** [build/makelib/common.mk:434: generate] Error 2

How can we reproduce it?

I encountered this when adding new resources to provider-aws. Starting from main, do the following:

  1. Add "aws_msk_cluster_policy": config.IdentifierFromProvider, to the terraform SDK external name config.
  2. Run make generate. It passes.
  3. Change the external name config to "aws_msk_cluster_policy": config.TemplatedStringAsIdentifier("", "{{ .parameters.cluster_arn }}"),
  4. Run make generate. It fails partway through with this error: 
    
17:31:50 [ .. ] verify go modules dependencies have expected content
all modules verified
17:31:52 [ OK ] go modules dependencies verified
17:31:52 [ .. ] generating provider schema for hashicorp/aws 5.31.0
17:31:55 [ OK ] generating provider schema for hashicorp/aws 5.31.0

...

17:31:57 [ .. ] go generate linux_amd64

Generated 933 resources! angryjet: error: error loading packages using pattern ./...: /home/matt/git/upbound-provider-aws/apis/kafka/v1beta1/zz_generated.resolvers.go:409:62: mg.Spec.InitProvider.ClusterArn undefined (type ScramSecretAssociationInitParameters has no field or method ClusterArn) exit status 1 apis/generate.go:34: running "go": exit status 1 17:35:45 [FAIL] make[1]: [build/makelib/golang.mk:240: go.generate] Error 1 make: [build/makelib/common.mk:434: generate] Error 2


This makes some sense, as the new external name configuration sets `ClusterArn` as an identifier field, which should not be in `Spec.InitProvider`, but I don't understand why angryjet is still looking for it there.
5. Run `make generate` again. It fails with the import cycle error above.

I can work around the issue by dropping the changes made by the first run of `make generate`, but that's not easy to do for existing resources, so one impact of this bug is it makes it very difficult to add an identifier field to an existing resource's external name configuration. Given that (at least in provider-aws) there are still many external name configs which are un-normalized, inconvenient to use, and undocumented, this is an area that I feel is important to be able to improve upon.
mbbush commented 5 months ago

I determined that the error comes from the resolvers. Running find . -name zz_generated.resolvers.go -delete before make generate prevents the angryjet error.