search

aws-amplify / amplify-category-api

The AWS Amplify CLI is a toolchain for simplifying serverless web and mobile development. This plugin provides functionality for the API category, allowing for the creation and management of GraphQL and REST based backends for your amplify project.
https://docs.amplify.aws/
Apache License 2.0
89 stars 79 forks source link

UPDATE_ROLLBACK_FAILED state and can not be updated. #2244

Open rafaelfaria opened 9 months ago

rafaelfaria commented 9 months ago

How did you install the Amplify CLI?

npm

If applicable, what version of Node.js are you using?

No response

Amplify CLI Version

12.10.1

What operating system are you using?

Mac

Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.

No

Describe the bug

I am not being able to deploy my project through CLI anymore.

when I try to deploy it now I get the following:

Stack:arn:aws:cloudformation:us-east-1:533267446078:stack/amplify-myapp-prod-54752/a3eccb60-bbba-11ee-9615-0a268650abe7 is in UPDATE_ROLLBACK_FAILED state and can not be updated.

Expected behavior

To push it properly.

Reproduction steps

I am not quite sure how I got to this state thou. However, the only thing I was fixing was that I had a custom Mutation pointing to the wrong lambda function

type Mutation {

  generateCaptcha(input: GennerateCaptchaInput): GenerateCaptchaResult
    @function(name: "GetCaptcha-${env}")
    @auth(rules: [{ allow: private, provider: iam }])
}

The following was wrong, since the lambda function was called GenerateCaptcha @function(name: "GetCaptcha-${env}")

When I tried to fix it with the correct name, everything stop working.

Now, it's hard to pinpoint that this was the problem, I got issues with "legacy" lamda layer references in my project, which caused issues too, not sure how this happens.

anyways, hard to pinpoint to something.

I am interested in how I can fix it. I have literally restarted my project and recreated EVERYTHING 3 times during my development due issues. I dont really want to do it again.

Project Identifier

No response

Log output

``` # Put your logs below this line ```

Additional information

No response

Before submitting, please confirm:

rafaelfaria commented 9 months ago

Looking at the stack I found this..

Embedded stack arn:aws:cloudformation:us-east-1:533267446078:stack/amplify-myapp-prod-54752-apiGraphQLAPI-44SP06RETH3B/07ba2540-c1c2-11ee-8d28-0a6786933c53 was not successfully updated. Currently in UPDATE_ROLLBACK_FAILED with reason: The following resource(s) failed to update: [FunctionDirectiveStack].

rafaelfaria commented 9 months ago

image

rafaelfaria commented 9 months ago

So to recap

i managed to deploy the wrong version

  generateCaptcha(input: GennerateCaptchaInput): GenerateCaptchaResult
    @function(name: "GetCaptcha-${env}")
    @auth(rules: [{ allow: private, provider: iam }])

Then i tried to fix the function name


  generateCaptcha(input: GennerateCaptchaInput): GenerateCaptchaResult
    @function(name: "GenerateCaptcha-${env}")
    @auth(rules: [{ allow: private, provider: iam }])

and that's when i started getting this issue.

image

ykethan commented 9 months ago

Hey,👋 thanks for raising this! I'm going to transfer this over to our API repository for better assistance 🙂

sundersc commented 9 months ago

Looks like the issue is caused by cross stack references. Could you try the below?

  1. Deploy the current state.

    generateCaptcha(input: GennerateCaptchaInput): GenerateCaptchaResult
@function(name: "GetCaptcha-${env}")
@auth(rules: [{ allow: private, provider: iam }])

  2. Remove the function directive and deploy.

    generateCaptcha(input: GennerateCaptchaInput): GenerateCaptchaResult
@auth(rules: [{ allow: private, provider: iam }])

  3. Deploy with the correct function name.

    generateCaptcha(input: GennerateCaptchaInput): GenerateCaptchaResult
@function(name: "GenerateCaptcha-${env}")
@auth(rules: [{ allow: private, provider: iam }])
rafaelfaria commented 9 months ago

@sundersc Thanks for your reply.

Unfortunately It doesn't allow me to redeploy.

is in UPDATE_ROLLBACK_FAILED state and can not be updated.

Every time I try to redeploy, this happens. It doesn't allow me to do anything from the CLI. I'm pretty much stuck and delayed on my project now.

Deploying resources into prod environment. This will take a few minutes. ⠹
Deploying root stack myapp [ ---------------------------------------- ] 0/17
Deploying api GraphQLAPI [ ---------------------------------------- ] 0/20
Deploying function myappAppLayer [ ---------------------------------------- ] 0/3
Deploying function MyAppAuthCreateAuthChallenge [ ---------------------------------------- ] 0/3
Deploying function MyAppAuthDefineAuthChallenge [ ---------------------------------------- ] 0/3
Deploying function MyAppAuthPostConfirmation [ ---------------------------------------- ] 0/4
Deploying function MyAppAuthPreSignup [ ---------------------------------------- ] 0/3
Deploying function MyAppAuthVerifyAuthChallengeResponse [ ---------------------------------------- ] 0/3
Deploying function MyAppCleanUpCaptcha [ ---------------------------------------- ] 0/6
Deploying function MyAppGenerateCaptcha [ ---------------------------------------- ] 0/4
🛑 Stack:arn:aws:cloudformation:us-east-1:533267446078:stack/amplify-myapp-prod-54752/a3eccb60-bbba-11ee-9615-0a268650abe7 is in UPDATE_ROLLBACK_FAILED state and can not be updated.

Learn more at: https://docs.amplify.aws/cli/project/troubleshooting/

DeploymentFault: Stack:arn:aws:cloudformation:us-east-1:533267446078:stack/amplify-myapp-prod-54752/a3eccb60-bbba-11ee-9615-0a268650abe7 is in UPDATE_ROLLBACK_FAILED state and can not be updated.
    at Object.run (/snapshot/amplify-cli/build/node_modules/@aws-amplify/amplify-provider-awscloudformation/lib/push-resources.js:362:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/extensions/amplify-helpers/push-resources.js:137:16
    at async Promise.all (index 0)
    at async providersPush (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/extensions/amplify-helpers/push-resources.js:133:5)
    at async AmplifyToolkit.pushResources (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/extensions/amplify-helpers/push-resources.js:107:13)
    at async Object.executeAmplifyCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/index.js:194:9)
    at async executePluginModuleCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/execution-manager.js:139:5)
    at async executeCommand (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/execution-manager.js:37:9)
    at async Object.run (/snapshot/amplify-cli/build/node_modules/@aws-amplify/cli-internal/lib/index.js:121:5)

Stack:arn:aws:cloudformation:us-east-1:533267446078:stack/amplify-myapp-prod-54752/a3eccb60-bbba-11ee-9615-0a268650abe7 is in UPDATE_ROLLBACK_FAILED state and can not be updated.
ValidationError: Stack:arn:aws:cloudformation:us-east-1:533267446078:stack/amplify-myapp-prod-54752/a3eccb60-bbba-11ee-9615-0a268650abe7 is in UPDATE_ROLLBACK_FAILED state and can not be updated.
    at Request.extractError (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/protocol/query.js:50:29)
    at Request.callListeners (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/request.js:686:14)
    at Request.transition (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/snapshot/amplify-cli/build/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /snapshot/amplify-cli/build/node_modules/aws-sdk/lib/state_machine.js:26:10
Deploying resources into prod environment. This will take a few minutes. ⠸
Deploying root stack myapp [ ---------------------------------------- ] 0/17
Deploying api GraphQLAPI [ ---------------------------------------- ] 0/20
Deploying function myappAppLayer [ ---------------------------------------- ] 0/3
Deploying function MyAppAuthCreateAuthChallenge [ ---------------------------------------- ] 0/3
Deploying function MyAppAuthDefineAuthChallenge [ ---------------------------------------- ] 0/3
Deploying function MyAppAuthPostConfirmation [ ---------------------------------------- ] 0/4
Deploying function MyAppAuthPreSignup [ ---------------------------------------- ] 0/3
Deploying function MyAppAuthVerifyAuthChallengeResponse [ ---------------------------------------- ] 0/3
Deploying function MyAppCleanUpCaptcha [ ---------------------------------------- ] 0/6
Deploying function MyAppGenerateCaptcha [ ---------------------------------------- ] 0/4

Session Identifier: 8b8724c6-ad64-4045-b54b-559c75011691

It looks like its missing a resolver. Can I manually recreate these resolver (and how?) and rerun the rollback?

image

I'm uncertain about the next steps to take. Is there a guide or runbook available for resolving this issue without the risk of losing my work?

This topic has been discussed in other threads, yet it seems there was no conclusive solution provided for when situations reach this point, particularly concerning CLI limitations that can solve this issue. Most discussions seem to end without a clear resolution. I'm hoping to find a way through this and perhaps be the exception by successfully addressing the issue.

I've come across suggestions to delete the cloud stack, establish a new environment, remove the production environment, and then execute amplify init. Would any of these approach be effective? Ex: This issue

rafaelfaria commented 9 months ago

@sundersc Any guidance on this matter? It's been a couple days that I am completely blocked.

rafaelfaria commented 9 months ago

I can't wait any longer, sorry. My app isn't up yet, so losing my data is okay for now. That's why I chose to delete and then set up my environment again, and it worked.

This fix wouldn't work if my app was live. It's a big risk for what I plan to do next. I've been using Amplify for a few years, and it's been tough. I've had to start over many times. Now, I'm thinking about whether to use Amplify for my next project.

I know Amplify is used for many things, but I've seen others have this problem too, without finding a solution or a guide to help. If this had happened when my app was live, it would have been a big problem.

Despite me solving my own problem, I know others will still encounter this in the future. So I hope it doesn't just get ignored and get this thread closed. I don't want to be just another case where no one really tries to find a solution that doesn't mean starting from scratch.

AleksandarGT commented 8 months ago

Of course this is still a problem that the developers of amplify know and won't fix..

fistofzen commented 2 months ago

Is there any update ?