Amplify Push with API Changes Randomly Fails

gakinson commented 3 years ago

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

How did you install the Amplify CLI?

npm

If applicable, what version of Node.js are you using?

12.20.1

Amplify CLI Version

4.44.2

What operating system are you using?

Mac

Amplify Categories

api

Amplify Commands

push

Describe the bug

Modified schema.graphql and did an "amplify push" and the following error appeared:

Following resources failed

Resource Name: cjxhd56bavemrbwvw2fljq2igyGraphQLSchema (AWS::AppSync::GraphQLSchema) Event Type: update Reason: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: Q9PRYHZG33HXFYQY; S3 Extended Request ID: XaPDoAbb024r5LxQY2MpXD/McDTGGPhX7/tqW3bAnTu2enrXXWR/s1yJSEiK2MJL6Z/SUGdglWM=; Proxy: null) URL: https://console.aws.amazon.com/cloudformation/home?region=us-east-2#/stacks/arn%3Aaws%3Acloudformation%3Aus-east-2%3A272407951964%3Astack%2Famplify-wesponsored-web-kinson-115740-apiwesponsoredweb-1O6ZVNYJYMS5W%2F62c9cbc0-2aa3-11eb-8513-0aff1dd11f0a/events

✖ An error occurred when pushing the resources to the cloud

Expected behavior

Push succeeded

Reproduction steps

Modify schema
amplify push

Log output

# Put your logs below this line

Following resources failed

Resource Name: cjxhd56bavemrbwvw2fljq2igyGraphQLSchema (AWS::AppSync::GraphQLSchema)
Event Type: update
Reason: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: Q9PRYHZG33HXFYQY; S3 Extended Request ID: XaPDoAbb024r5LxQY2MpXD/McDTGGPhX7/tqW3bAnTu2enrXXWR/s1yJSEiK2MJL6Z/SUGdglWM=; Proxy: null)
URL: https://console.aws.amazon.com/cloudformation/home?region=us-east-2#/stacks/arn%3Aaws%3Acloudformation%3Aus-east-2%3A272407951964%3Astack%2Famplify-wesponsored-web-kinson-115740-apiwesponsoredweb-1O6ZVNYJYMS5W%2F62c9cbc0-2aa3-11eb-8513-0aff1dd11f0a/events

✖ An error occurred when pushing the resources to the cloud

If I made additional edits and tried, eventually it worked. Error seems random because it happened more than once.

edwardfoyle commented 3 years ago

Hi @gakinson I'm not able to reproduce this on my machine. If you figure out any additional steps that consistently leads to this failure please follow up with those details.

Raees678 commented 3 years ago

Hi, I can confirm I'm having a similar issue on my Amplify projects as well. I cant seem to reproduce it either, but can confirm it recently started popping up in multiple amplify environments for us.

fossamagna commented 3 years ago

I could reproduce this issue. but in my case not modfiy schema.

Reproduction steps

Successful deployment with amplify push (versions < 4.40.0, 4.29.1 in my case)
Remove api/<resource_name>/build directory.
Run amplify push again (versions >= 4.40.0, 4.47.1 in my case).
NoSuchKey Error

When execute above steps, I see amplify resource table on terminal in step 2 follow to:

Category	Resource name	Operation	Provider plugin
Auth	fooAuth	Update	awscloudformation
Api	fooApi	No Change	awscloudformation

Api category is No Change operation, but CloudFormation be ran by Update operation of Auth category in CloudFormation template. Since the version of amplify cli has been upgraded, the version of Node.js in the Auth category will be changed from nodejs v10 to nodejs v12.x, and it will be automatically Update operation.

Because of Api category is No Change operation, no upload appsync files (ex: schema.graphql, *.vtl, ...). Usually in such cases, hash diretory is same previous deployment. So uploaded appsync files to S3 in previous deployment is used in CloudFormation.

In amplify push again step, hash directory of previous deployment is computed from amplify/#current-cloud-backend/api/<resource_name>. on the other hand, hash directory of this time deployment is computed from amplify/backend/api/<resource_name>. expect same hash because of api category is no change. But In this case hash directory is not same. The cause is that amplify/backend/api/<resource_name>/parameters.json is updated during the build. At previous deployment, The Indent is 4 space in parameters.json, but it is updated 2 spaceis during the build. So, hash is not same. As a result, Resources of expected key in CloudFormation template of Auth category is not in S3 bucket.

I presume that the cause was that the indent was changed by the following correction. https://github.com/aws-amplify/amplify-cli/pull/6133/files#diff-1e0f7f515594c0ffce7fe89d3d7b71e373e2673066b49b28b3fd9b353adeebbcL483-L487

fossamagna commented 3 years ago

Workaround: amplify push --force

With --force option, You can force upload appsync files to S3 bucket.

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] commented 3 years ago

This issue has been automatically locked since there hasn't been any recent activity after it was closed. Please open a new issue for related bugs.

Looking for a help forum? We recommend joining the Amplify Community Discord server *-help channels for those types of questions.

aws-amplify / amplify-cli