Backend build not working(timed out)

[jothikannan89]

Before opening, please confirm:

• [X] I have checked to see if my question is addressed in the FAQ.
• [X] I have searched for duplicate or closed issues.
• [X] I have read the guide for submitting bug reports.
• [X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

App Id

d1yvu27h0gqugj

Region

us-east-2

Amplify Console feature

Backend builds

Describe the bug

I have my frontend repo in BitBucket and using another amplify backend by amplify pull . In local the app is working fine. I am trying to host the frontend, So selected BitBukcet hosting environment and selected my repo and followed "Automatic build-time generation of Amplify config " to deploy.

But it taking too much time for the deploy, after 30mins got Timeout issue

Expected behavior

As the Frontend is working fine in local, the deployment should work here. But the deployment is not happening for last 3 days.

Reproduction steps

1. Create react app
2. Pull existing amplify backend by using amplify pull --appId ##### --envName dev
3. Push the code into Bitbucket repo
4. Host the repo in amplify frontend environment with Automatic build-time generation

Build Settings

version: 1
backend:
  phases:
    build:
      commands:
        - '# Execute Amplify CLI with the helper script'
        - amplifyPush --simple
frontend:
  phases:
    preBuild:
      commands:
        - npm ci
    build:
      commands:
        - npm run build
  artifacts:
    baseDirectory: build
    files:
      - '**/*'
  cache:
    paths:
      - node_modules/**/*

Additional information

[github-actions[bot]]

Hi :wave:, thanks for opening! While we look into this...

If this issue is related to custom domains, be sure to check the custom domains troubleshooting guide to see if that helps. Also, there is a more general troubleshooting FAQ that may be helpful for other questions.

Lastly, please make sure you've specified the App ID and Region in the issue!

[ghost]

Hi @jothikannan89 👋🏽, thank you for raising this issue. Did you happen to delete this app?

[dylan-westbury]

@hloriana

I am facing this issue also, build just keeps failing due to timeout. I have tried 2 branches with the same backend (staging), disconnect and reconnect. I also re-select the branch to build the env.

App ID d1wuh42i6cy93b

At once stage many new backends were created for one of the branch, could have been my fault, but it was timing out for new env also.

The last build message for backend where it gets stuck

Building resource auth/userPoolGroups

[ghost]

Hi @dylan-westbury what region is your app deployed in?

[dylan-westbury]

@hloriana

us-east-1.

I believe the issue is, AWS created duplicated pinpoint resources, I removed one thinking it wasn't in use but it actually was.

https://github.com/aws-amplify/amplify-cli/issues/5079

After removing completely and re-adding it resolved

[ghost]

Closing due to no response. @jothikannan89 please re-open if you run into this issue again.

[dylan-westbury]

Hi @hloriana

We are facing this issue again.

When we trigger a CI / CD build in amplify hosting, the backend deployment is stuck on

2022-10-20T02:30:09.397Z [WARNING]: - Building resource auth/userPoolGroups

After 30 minutes it times out.

We were able to deploy the backend locally on computer via amplify push, however just for amplify hosting it is stuck.

We also tried to skip the push within amplify hosting following the documentation here https://docs.aws.amazon.com/amplify/latest/userguide/amplify-config-autogeneration.html

However it still gets stuck:

2022-10-20T02:29:57.710Z [INFO]: ## Running amplify pull to generate aws-exports.js file for frontend
2022-10-20T02:30:01.399Z [INFO]: Amplify AppID found: d3tyae2uog3myc. Amplify App name is: XXXXX
2022-10-20T02:30:01.464Z [INFO]: Backend environment prod found in Amplify Console app: Swysh Backoffice
2022-10-20T02:30:06.000Z [WARNING]: - Fetching updates to backend environment: prod from the cloud.
2022-10-20T02:30:09.397Z [WARNING]: - Building resource auth/userPoolGroups

We are facing a blocker as can't deploy our frontend for production, is there any way we can get around this issue? Amplify pull / push work on our local but just not amplify hosting.

App ID d3tyae2uog3myc if that helps

We also can't remove pinpoint from production as this contains all our user's endpoints, in which they receive vital push notifications.

[Etep15]

I've run into this as well for the past few weeks when deploying from our dev branch to our test branch for deployment to our test env.

Stuck at the exact same spot, Build resource auth/... Then times out after 30 minutes.

Using the CLI amplify push works fine.

I thought I had it sorted last week by doing the following...

1. amplify notifications remove (removed our APNS setup)
2. amplify analytics remove
3. amplify push
4. amplify notifications remove (just for good measure to remove the pinpoint application if it's still there)
5. check the web console and ensure that there is no application listed for the env (in our case test) in the Pinpoint service.
6. amplify analytics add
7. amplify push (important step to get the Pinpoint application setup in AWS).
8. amplify notifications add

At this point checking Pinpoint on the web console should show only 1 entry for the environment (if step 7 wasn't done there would end up being two).

After I did this, things were working, last week. However when I did a merge to our test branch tonight, the problem came back. Steps I took to merge :

1. git checkout test
2. amplify env checkout test
3. git merge main (resolve conflicts and always choose the settings that were in the test branch before the merge for the amplify json files that come up as conflicts).
4. amplify env checkout test
5. git commit
6. git push

This kicks off the CI/CD but somehow the Pinpoint bug is back. I confirmed that amplify/backend/analytics/myapp/parameters.json is the same as what it was when it last worked.

I just tried my fix from above and unfortunately it is not working this time.

Amplify cli version 10.3.0

Looking for help here as well.

[ghost]

Hi @dylan-westbury @Etep15 sorry to hear you are experiencing this issue. We are actively investigating this and will update you shortly.

[ghost]

@dylan-westbury @Etep15 we are still working on reproducing this behavior on our end but we'd like to unblock you as soon as possible. It appears that you're being prompted to select an authentication method (certificate or key) for APNs which is causing the build to time out. To work around this, can you try modifying the backend build phase of your amplify.yml to:

version: 1
backend:
  phases:
    build:
      commands:
        - '# Execute Amplify CLI with the helper script'
        - amplifyPush --simple --yes

The --yes flag should allow the script to run in headless mode to prevent prompts during the build. Headless mode for CI/CD. Please let us know if this unblocks you.

[dylan-westbury]

Hi @hloriana

Thanks for coming back to us. For the work around, we are a bit worried this could disable APNS push notifications, do you know if this could be the case or would it skip it altogether? I guess we could resolve this directly in pinpoint UI if it does?

[giorgiocarniel]

Had the same problem today (we use pinpoint too)

Solved by setting a fixed amplify cli version under App Settings / Build settings

If it's something pinpoint related I would suggest to add some kind of verbosity, I was investigating any Cognito related changes due to the last logging line.

[kjsieffert]

Our Amplify build started failing today with this exact same issue. It hangs at Building resource auth/userPoolGroups

Setting the version back to 10.2.3 as noted by giorgiocarniel solved the issue, at least until a more permanent fix is identified.

Then ran into a Javascript of of memory error during the Frontend build. Fixed this by adding:

• export NODE_OPTIONS=--max-old-space-size=8192

[Etep15]

Our Amplify build started failing today with this exact same issue. It hangs at Building resource auth/userPoolGroups

Setting the version back to 10.2.3 as noted by giorgiocarniel solved the issue, at least until a more permanent fix is identified.

Then ran into a Javascript of of memory error during the Frontend build. Fixed this by adding:

• export NODE_OPTIONS=--max-old-space-size=8192

@kjsieffert Where did you add this option?

[Etep15]

@dylan-westbury @Etep15 we are still working on reproducing this behavior on our end but we'd like to unblock you as soon as possible. It appears that you're being prompted to select an authentication method (certificate or key) for APNs which is causing the build to time out. To work around this, can you try modifying the backend build phase of your amplify.yml to:

version: 1
backend:
  phases:
    build:
      commands:
        - '# Execute Amplify CLI with the helper script'
        - amplifyPush --simple --yes

The --yes flag should allow the script to run in headless mode to prevent prompts during the build. Headless mode for CI/CD. Please let us know if this unblocks you.

@hloriana I appreciate the response, thank you! Unfortunately this didn't help at all. Still stuck on Building resource auth/...

Going to try some of the other suggestions to reduce the amplify cli version.

[Etep15]

I can confirm that @giorgiocarniel solution solves the issue. I guess as the new amplify cli version was updated the build then used the "latest" version. Setting it to 10.2.3 allows things to proceed. I didn't need to do the change that @kjsieffert mentioned. Only needed to change the cli version in the web console.

This is a temporary fix obviously so looking forward to know when I can switch it back to latest after the cli is fixed.

Thanks again all!

[gspeicher]

I know I already said this in the office hours Discord chat room, but I still feel like there is too much of a coincidence that CLI 10.3.0 is now prompting for Notifications configuration values on every interactive checkout/pull, and meanwhile 10.3.0 headless pulls are timing out. Seems like the headless pull is waiting for that value even though the prompt is not visible in the logs.

[Etep15]

I know I already said this in the office hours Discord chat room, but I still feel like there is too much of a coincidence that CLI 10.3.0 is now prompting for Notifications configuration values on every interactive checkout/pull, and meanwhile 10.3.0 headless pulls are timing out. Seems like the headless pull is waiting for that value even though the prompt is not visible in the logs.

Yes I agree. I thought it may be in a prior version of amplify cli as well (10.2.x). But because my amplify app settings/build settings was set to "latest" (default) for the cli to use when building, I agree that 10.3.0 must be the culprit. Even though I hadn't updated my local version, the version used by CI/CD was the latest.

[ghost]

It appears that this issue is related to Amplify CLI version 10.3.0 so please use an earlier version of the CLI to resolve this behavior. For anyone else running into this issue please defer to https://github.com/aws-amplify/amplify-cli/issues/11238 for further assistance.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.