aws-cloudformation / cloudformation-cli

The CloudFormation Provider Development Toolkit allows you to author your own resource providers and modules that can be used by CloudFormation.
Apache License 2.0
316 stars 157 forks source link

`make` doesn't work cloudformation-cli latest versions causing `cfn submit` to fail #1053

Open maastha opened 5 months ago

maastha commented 5 months ago

Hi

We are running into errors when submitting a CFN resource to private registry. Essentially it looks like an issue with the make command in cloudformation-cli.

We noticed that the issue does not occur in below versions: cloudformation-cli==0.2.33 cloudformation-cli-go-plugin==2.0.4

The issue occurs if the latest versions of the CLI are used i.e. cloudformation-cli v0.2.34 and v0.2.35 and cloudformation-cli-go-plugin v2.2.0

Expected Behavior:

Actual Behavior:


rpdk.log:

[2023-12-29T11:23:40Z] DEBUG - inputs_1_update.json found. Writing to package. [2023-12-29T11:23:40Z] INFO - Packaging Go project [2023-12-29T11:23:40Z] DEBUG - No Go executable match [2023-12-29T11:23:40Z] DEBUG - Caught exit recommendation Traceback (most recent call last): File "/opt/homebrew/lib/python3.11/site-packages/rpdk/core/cli.py", line 105, in main args.command(args) File "/opt/homebrew/lib/python3.11/site-packages/rpdk/core/submit.py", line 19, in submit project.submit( File "/opt/homebrew/lib/python3.11/site-packages/rpdk/core/project.py", line 652, in submit self._add_resources_content_to_zip(zip_file) File "/opt/homebrew/lib/python3.11/site-packages/rpdk/core/project.py", line 686, in _add_resources_content_to_zip self._plugin.package(self, zip_file) File "/opt/homebrew/lib/python3.11/site-packages/rpdk/go/codegen.py", line 281, in package self._find_exe(project) File "/opt/homebrew/lib/python3.11/site-packages/rpdk/go/codegen.py", line 257, in _find_exe raise GoExecutableNotFoundError( rpdk.go.codegen.GoExecutableNotFoundError: You must build the handler before running cfn-submit. Please run 'make' or the equivalent command in your IDE to compile and package the code. [2023-12-29T11:23:40Z] CRITICAL - You must build the handler before running cfn-submit. Please run 'make' or the equivalent command in your IDE to compile and package the code. [2023-12-29T14:54:34Z] DEBUG - Logging set up successfully [2023-12-29T14:54:34Z] DEBUG - Running generate: Namespace(version=False, subparser_name='generate', command=<function generate at 0x12244ce00>, verbose=0, endpoint_url=None, region=None, local_only=False, target_schemas=[], profile=None) [2023-12-29T14:54:34Z] DEBUG - Root directory: /Users/aastha.mahendru/mongodbatlas-cloudformation-resources/cfn-resources/network-container [2023-12-29T14:54:34Z] DEBUG - Loading project file '/Users/aastha.mahendru/mongodbatlas-cloudformation-resources/cfn-resources/network-container/.rpdk-config' [2023-12-29T14:54:34Z] INFO - Validating your resource specification... [2023-12-29T14:54:34Z] DEBUG - Rewriting refs in '' (file:///Users/aastha.mahendru/mongodbatlas-cloudformation-resources/cfn-resources/network-container/mongodb-atlas-networkcontainer.json) [2023-12-29T14:54:34Z] WARNING - Resource schema is valid. [2023-12-29T14:54:34Z] INFO - Validating your resource schema... [2023-12-29T14:54:34Z] DEBUG - Writing Execution Role CloudFormation template: /Users/aastha.mahendru/mongodbatlas-cloudformation-resources/cfn-resources/network-container/resource-role.yaml [2023-12-29T14:54:34Z] DEBUG - Overwriting '/Users/aastha.mahendru/mongodbatlas-cloudformation-resources/cfn-resources/network-container/resource-role.yaml' [2023-12-29T14:54:34Z] DEBUG - Generate started [2023-12-29T14:54:34Z] DEBUG - Writing Types [2023-12-29T14:54:34Z] DEBUG - Overwriting '/Users/aastha.mahendru/mongodbatlas-cloudformation-resources/cfn-resources/network-container/cmd/resource/config.go' [2023-12-29T14:54:34Z] DEBUG - Overwriting '/Users/aastha.mahendru/mongodbatlas-cloudformation-resources/cfn-resources/network-container/cmd/resource/model.go' [2023-12-29T14:54:34Z] DEBUG - Writing project: /Users/aastha.mahendru/mongodbatlas-cloudformation-resources/cfn-resources/network-container/cmd/main.go [2023-12-29T14:54:34Z] DEBUG - Overwriting '/Users/aastha.mahendru/mongodbatlas-cloudformation-resources/cfn-resources/network-container/cmd/main.go' [2023-12-29T14:54:34Z] DEBUG - Writing makebuild: /Users/aastha.mahendru/mongodbatlas-cloudformation-resources/cfn-resources/network-container/makebuild [2023-12-29T14:54:34Z] DEBUG - Overwriting '/Users/aastha.mahendru/mongodbatlas-cloudformation-resources/cfn-resources/network-container/makebuild' [2023-12-29T14:54:34Z] DEBUG - Removing generated docs: /Users/aastha.mahendru/mongodbatlas-cloudformation-resources/cfn-resources/network-container/docs [2023-12-29T14:54:34Z] DEBUG - Writing generated docs [2023-12-29T14:54:34Z] DEBUG - Finished documenting nested properties [2023-12-29T14:54:34Z] DEBUG - Writing docs README: /Users/aastha.mahendru/mongodbatlas-cloudformation-resources/cfn-resources/network-container/docs/README.md [2023-12-29T14:54:34Z] WARNING - Generated files for MongoDB::Atlas::NetworkContainer [2023-12-29T14:54:34Z] DEBUG - Finished generate [2023-12-29T14:54:42Z] DEBUG - Logging set up successfully [2023-12-29T14:54:42Z] DEBUG - Running submit: Namespace(version=False, subparser_name='submit', command=<function submit at 0x1168e1440>, verbose=0, dry_run=False, endpoint_url=None, region='eu-south-1', set_default=False, profile=None, role_arn=None, use_role=True, use_docker=False, no_docker=False) [2023-12-29T14:54:42Z] DEBUG - Root directory: /Users/aastha.mahendru/mongodbatlas-cloudformation-resources/cfn-resources/network-container [2023-12-29T14:54:42Z] DEBUG - Loading project file '/Users/aastha.mahendru/mongodbatlas-cloudformation-resources/cfn-resources/network-container/.rpdk-config' [2023-12-29T14:54:42Z] INFO - Validating your resource specification... [2023-12-29T14:54:42Z] DEBUG - Rewriting refs in '' (file:///Users/aastha.mahendru/mongodbatlas-cloudformation-resources/cfn-resources/network-container/mongodb-atlas-networkcontainer.json) [2023-12-29T14:54:42Z] WARNING - Resource schema is valid. [2023-12-29T14:54:42Z] INFO - Validating your resource schema... [2023-12-29T14:54:42Z] DEBUG - inputs_1_create.json found. Writing to package. [2023-12-29T14:54:42Z] DEBUG - inputs_1_update.json found. Writing to package. [2023-12-29T14:54:42Z] INFO - Packaging Go project [2023-12-29T14:54:42Z] DEBUG - No Go executable match [2023-12-29T14:54:42Z] DEBUG - Caught exit recommendation Traceback (most recent call last): File "/opt/homebrew/lib/python3.11/site-packages/rpdk/core/cli.py", line 105, in main args.command(args) File "/opt/homebrew/lib/python3.11/site-packages/rpdk/core/submit.py", line 19, in submit project.submit( File "/opt/homebrew/lib/python3.11/site-packages/rpdk/core/project.py", line 652, in submit self._add_resources_content_to_zip(zip_file) File "/opt/homebrew/lib/python3.11/site-packages/rpdk/core/project.py", line 686, in _add_resources_content_to_zip self._plugin.package(self, zip_file) File "/opt/homebrew/lib/python3.11/site-packages/rpdk/go/codegen.py", line 281, in package self._find_exe(project) File "/opt/homebrew/lib/python3.11/site-packages/rpdk/go/codegen.py", line 257, in _find_exe raise GoExecutableNotFoundError( rpdk.go.codegen.GoExecutableNotFoundError: You must build the handler before running cfn-submit. Please run 'make' or the equivalent command in your IDE to compile and package the code. [2023-12-29T14:54:42Z] CRITICAL - You must build the handler before running cfn-submit. Please run 'make' or the equivalent command in your IDE to compile and package the code.

kddejong commented 5 months ago

@maastha are you running make before running cfn submit? make should result in a bin/bootstrap file to be created and that file needs to exist for cfn submit to work.

https://github.com/aws-cloudformation/cloudformation-cli-go-plugin/blob/a96dc21b798d1d23e7d1fb7cd6e17b4f0fc96362/python/rpdk/go/codegen.py#L254-L271

maastha commented 5 months ago

@kddejong yes I am running make before cfn submit but still running into this issue:

➜  network-container git:(CLOUDP-221421-organization-new-params) ✗ make
cfn generate
Resource schema is valid.
Generated files for MongoDB::Atlas::NetworkContainer
env GOOS=linux CGO_ENABLED=0 GOARCH=amd64 go build -ldflags="-s -w -X github.com/mongodb/mongodbatlas-cloudformation-resources/util.defaultLogLevel=info -X github.com/mongodb/mongodbatlas-cloudformation-resources/version.Version=2bf029c4df9cea1bafedd6c25aa73123bc40a3df" -tags="logging callback metrics scheduler" -o bin/handler cmd/main.go

➜  network-container git:(CLOUDP-221421-organization-new-params) ✗ cfn submit --region eu-south-1
Resource schema is valid.
You must build the handler before running cfn-submit.
Please run 'make' or the equivalent command in your IDE to compile and package the code.

I see a bin/handler is generated image

I think it could be an issue with the go-plugin as well because I was able to run make and then cfn submit with cfn cli version 0.2.35 and go-plugin version 2.0.4 but not with go-plugin version 2.2.0

➜  network-container git:(CLOUDP-221421-organization-new-params) ✗ cfn --version                 
cfn 0.2.35
kddejong commented 5 months ago

PR https://github.com/aws-cloudformation/cloudformation-cli-go-plugin/pull/230 changed it. The code is looking for bin/bootstrap not bin/handler. Looks like we need to update the makefile to have it building a bin/bootstrap file.

https://github.com/aws-cloudformation/cloudformation-cli-go-plugin/blob/master/python/rpdk/go/templates/makebuild

kddejong commented 5 months ago

cfn generate will also recreate the makebuild file