search

aws-amplify / amplify-hosting

AWS Amplify Hosting provides a Git-based workflow for deploying and hosting fullstack serverless web applications.
https://aws.amazon.com/amplify/hosting/
Apache License 2.0
441 stars 113 forks source link

Deploy fails with "[ERROR]: Failed to deploy" #115

Closed desertmonkey closed 3 years ago

desertmonkey commented 4 years ago

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

  1. Start a build on our master branch
  2. See error during the Deploy step.

Expected behavior The build should succeed, as it has every other time before today.

Additional context We have a stage branch and a master branch in Amplify. We merge GitHub release branches into the stage branch first to verify it all looks good. The builds on stage work flawlessly. Then we merge that same release branch into the master branch and the build fails to deploy.

I thought this could be related to this issue but I reduced the size of our build by a couple hundred MBs, and it still fails to deploy.

cawleyflower commented 4 years ago

@desertmonkey and I have been solving this issue This deploy bug has been persistent in our Amplify app for a while now. I first noticed when trying to set up a new branch with Amplify, and were recieving this same build and deploy error consistently. It had never affected the master (production) branch until today.

I tried a number of fixes to get this working. We removed some duplicate binary files, which cut the build artifact down by half. This alone didn't fix the deploy error.

I finally have a fix in to get the master branch up and running. We are on a old version of Hugo, and our build requires an extended version to build. I moved the wget and installation Build Commands into the Prebuild, and the app started to deploy.

The bug still persists, and could have been quite costly.

swaminator commented 4 years ago

@cawleyflower thanks for providing the update. We have a known issue with deployment sizes. We'll update you when the fix is rolled out to all regions.

KununguaK-Iversen commented 4 years ago

AWS amplify console completes provision & build stages, but fails at deploy without giving a reason... The last 3 lines of my deployment log:

2019-09-23T22:56:23 [INFO]: Updating Edge config 2019-09-23T22:56:23 [ERROR]: Failed to deploy 2019-09-23T22:56:23 [INFO]: Deployment finished.

The crazy thing is i can't even re-deploy a previously stable build. What's up ? My App ARN: arn:aws:amplify:eu-west-1:352247527096:apps/d1p38jyf52tcwc

Weicocu commented 4 years ago

Same exact issue. My project is pretty simple, and my zipped build artifact is exactly 8000 KB, which seems suspicious. The last build that deployed successfully has a 7998 KB build artifact.

Looks similar to this issue, but they said something about a 1 GB build artifact limit. So my 8MB limit seems crazy small!

swaminator commented 4 years ago

146

niallmc commented 4 years ago

I am having the same issue:

2019-09-24T00:04:49 [INFO]: Updating Edge config 2019-09-24T00:04:49 [ERROR]: Failed to deploy 2019-09-24T00:04:49 [INFO]: Deployment finished.

swaminator commented 4 years ago

Can you try now? A fix has been rolled out.

Weicocu commented 4 years ago

Fix worked for me, thank you!!

ghost commented 4 years ago

I have same issue. So my issue didn't solve yet. I use ap-northeast-1 region. Could you tell me the correct version of Amplify Cli? My app size is about 500MB. It's smaller than 1GB. So, if amplify were fixed already, my app would be deployed...

cawleyflower commented 4 years ago

@swaminator We saw the issue go away, but now that our build size has reached 500 MB, we are failing to deploy.

cawleyflower commented 4 years ago

@swaminator is there any way we can have our deploy size limit increased as it states here #41 ?

Our ARN is here: arn:aws:amplify:us-west-2:054500231008:apps/d1ev2ep2azk4pn

thys-unomena commented 4 years ago

Can you try now? A fix has been rolled out.

Thank you very much!

Has this been rolled out for the region 'us-west-2' also? I'm still seeing this issue. Even now that our build artifact size was reduced from 500 MB to 26 MB

swaminator commented 4 years ago

@thys-unomena can you share your appid?

cawleyflower commented 4 years ago

@swaminator @thys-unomena is referring to the same app with ARN: arn:aws:amplify:us-west-2:xxxxxxxxxxxxxx:apps/d1ev2ep2azk4pn

thys-unomena commented 4 years ago

@swaminator @thys-unomena is referring to the same app with ARN: arn:aws:amplify:us-west-2:054500231008:apps/d1ev2ep2azk4pn

Hi @swaminator Did you do something on your side? About 10 hours ago the builds started deploying successfully.

If it was something that you did, can you please provide some kind of an explanation? And how can we prevent this from happening again?

rickiesmooth commented 4 years ago

I'm having the same problem since I've added cypress E2E tests

JenniferMcIntyre-xyz commented 4 years ago

Was this solved in #101?

I'm still getting the same error on my staging and prod environments. I'm going to take out some of the assets that I no longer use to see if it makes a significant change.

pcoleman-pixafy commented 4 years ago

@swaminator this is happening on the staging-mx branch of my build as well:

ARN: arn:aws:amplify:us-east-2:550189166758:apps/d1wxird61210hn

My team and I have 3 other branches that deploy fine without issue.

dzincone commented 4 years ago

Having the same issue on my 'dev' branch

ARN: arn:aws:amplify:us-east-2:392911942550:apps/dggeq6y7mtf3z

rickiesmooth commented 4 years ago

does it help putting your ARN number here?

JenniferMcIntyre-xyz commented 4 years ago

It was a cache size error for me. I solved it by removing the branch and re-adding it.

dzincone commented 4 years ago

My issue was solved by correcting the build path for the build step. Was defaulted to / and needed it at /build

rickiesmooth commented 4 years ago

weird, I re-added my branch (by removing and adding the app?) and as far as I can see my build path is correctly setup (for me it's web-build). Would be great if someone from the amplify team could advise.

rickiesmooth commented 4 years ago

@swaminator maybe you have an idea to reset the cache or something? I'm having this issue for a while now since I've added cypress tests.

litwicki commented 4 years ago

@rickiesmooth forgive the obvious question, but if you disable Cyprus are you able to build?

rickiesmooth commented 4 years ago

@litwicki thanks for getting back to me, yes I've tried to disable all test but that didnt help unfortunately. I did notice that when I try to amplify publish it locally it hangs on uploading my build artifacts

Index document is currently set as:
  index.html

⠏ Uploading files...
bordeo commented 4 years ago

@swaminator I have the same problem with my app d1ll0ojvkzv9e7

Could you help me?

Zelig880 commented 4 years ago

Hi guys,

I have the same issue. The App id is: d10w1ixpj82f3d

To provide you more details:

nimacks commented 4 years ago

We're actively investigating issues related to builds failing.

nimacks commented 4 years ago

I have the same issue. The App id is: d10w1ixpj82f3d @Zelig880 , can you provide the region your app is in?

nimacks commented 4 years ago

I have the same problem with my app d1ll0ojvkzv9e7 @bordeo , can you provide the region your app is in?

nimacks commented 4 years ago

@rickiesmooth , can you provide your appId and region?

bordeo commented 4 years ago

@nimacks eu-west-1

Zelig880 commented 4 years ago

Us West (Oregon)

On Fri, 8 Nov 2019, 5:33 pm Samuel Mensah, notifications@github.com wrote:

I have the same issue. The App id is: d10w1ixpj82f3d @Zelig880 https://github.com/Zelig880 , can you provide the region your app is in?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/aws-amplify/amplify-console/issues/115?email_source=notifications&email_token=ACE7HAR65P7CEQBLAHNXNYLQSWPHXA5CNFSM4IWULLI2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDS2FEY#issuecomment-551920275, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACE7HATEVQL7IZB5CD54OZDQSWPHXANCNFSM4IWULLIQ .

nimacks commented 4 years ago

@bordeo, it looks like your build is failing here are some logs :

2019-11-08T11:16:17.817Z [INFO]: $ gatsby build 
2019-11-08T11:16:19.689Z [INFO]: success open and validate gatsby-configs — 0.071 
2019-11-08T11:16:20.353Z [INFO]: success load plugins — 0.601 
2019-11-08T11:16:20.361Z [INFO]: success onPreInit — 0.008 
2019-11-08T11:16:22.069Z [INFO]: success delete html and css files from previous builds — 1.708 
2019-11-08T11:16:22.086Z [INFO]: success initialize cache — 0.017 
2019-11-08T11:16:22.118Z [INFO]: success copy gatsby files — 0.031 
2019-11-08T11:16:22.136Z [WARNING]: error 
                                    "bucketName" is a required option for gatsby-plugin-s3 
                                    See docs here - https://github.com/jariz/gatsby-plugin-s3 
             See our docs page for more info on this error: https://gatsby.dev/issue-how-to 
2019-11-08T11:16:22.210Z [WARNING]: error Command failed with exit code 1. 
2019-11-08T11:16:22.210Z [INFO]: info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command. 
2019-11-08T11:16:22.218Z [ERROR]: !!! Build failed
bordeo commented 4 years ago

@nimacks this is the last build. I had errore on build 69/70. I don’t remember. The error are on deploy step

bordeo commented 4 years ago

@nimacks the build on deploy error is 64

Screenshot 2019-11-11 at 11 54 12

Zelig880 commented 4 years ago

@nimacks I do apologies, I have provided you the wrong region above.

My region is US East (N. Virginia).

I have just tried to deploy one of the failing branch again, and it is still failing. I will try now to create a new Branch, just in case the problem has been solved elsewhere.

Please keep us updated on the progress and the cause behind this issue, and it is affecting project delivery!

nimacks commented 4 years ago

We have identified an issue in which customers with a very large set of files are unable to deploy their sites. We are actively prioritizing a fix for this. In the meantime, customers can delete unused branches in their app as a workaround.

rickiesmooth commented 4 years ago

I'm still seeing this issue and I believe I have only one branch? My appId is dmm4bu2tim2c and the region is eu-central-1, hope it helps!

rkul commented 4 years ago

We have just 9 PRs opened and the deployments fail randomly now with the same message as above:

2019-11-18T13:19:48 [INFO]: Updating Edge config
2019-11-18T13:19:48 [ERROR]: Failed to deploy
2019-11-18T13:19:48 [INFO]: Deployment finished.

Application ARN is arn:aws:amplify:us-east-1:044883579400:apps/d2lwxktxnpxork

@nimacks could you provide any ETA or workaround? Our main frontend application in production is affected.

ihao8 commented 4 years ago

Hello @rkul , you could try:

  1. If you have any branches that aren't being actively used, you can disconnect those branches which should be able to unblock you.
  2. Connect a new app.

If you still have this issue, please let us know.

ihao8 commented 4 years ago

Hello @rickiesmooth , I checked your build on Nov 17, the reason of deploy failed is: (node:49) UnhandledPromiseRejectionWarning: Error: ENOENT: no such file or directory, open '/codebuild/output/src366734173/src/home/../testConfig.json'

Could you double check if the file or directory exist?

rickiesmooth commented 4 years ago

@Joycehao19 thanks for looking into this! I believe this file was used for cypress, but that's not on master anymore. So I'm not sure which version of my code it's trying to build there..

kkesley-shine commented 4 years ago

we have the same problem currently.. our app id is d3rfa777e3c1ou in ap-southeast-2.

don't know if this is relevant,

but we noticed that the error comes when we changed the cypress version in yarn lock. If we use 3.4.1, the tests are running. (sometimes it freezes but it's running). When we upgraded our cypress version to 3.6.1 in package.json, it fixes the freeze.

But when we changed our yarn.lock to include that 3.6.1, it doesn't even run the test. These are the last log from test tab

cypress run --reporter mochawesome --reporter-options "reportDir=cypress/report/mochawesome-report,overwrite=false,html=false,json=true,timestamp=mmddyyyy_HHMMss"

and then it goes straight to deployment

2019-11-21T04:40:01 [INFO]: Starting Deployment 2019-11-21T04:40:01 [ERROR]: Failed to deploy 2019-11-21T04:40:01 [INFO]: Deployment finished.

Update

I can confirm that the cypress version in yarn.lock is causing it.

Because when I rollback to:

cypress@3.4.1:
  version "3.4.1"
  resolved "https://registry.yarnpkg.com/cypress/-/cypress-3.4.1.tgz#ca2e4e9864679da686c6a6189603efd409664c30"
  integrity sha512-1HBS7t9XXzkt6QHbwfirWYty8vzxNMawGj1yI+Fu6C3/VZJ8UtUngMW6layqwYZzLTZV8tiDpdCNBypn78V4Dg==
  dependencies:
    "@cypress/listr-verbose-renderer" "0.4.1"
    "@cypress/xvfb" "1.2.4"
    arch "2.1.1"
    bluebird "3.5.0"
    cachedir "1.3.0"
    chalk "2.4.2"
    check-more-types "2.24.0"
    commander "2.15.1"
    common-tags "1.8.0"
    debug "3.2.6"
    execa "0.10.0"
    executable "4.1.1"
    extract-zip "1.6.7"
    fs-extra "5.0.0"
    getos "3.1.1"
    is-ci "1.2.1"
    is-installed-globally "0.1.0"
    lazy-ass "1.6.0"
    listr "0.12.0"
    lodash "4.17.15"
    log-symbols "2.2.0"
    minimist "1.2.0"
    moment "2.24.0"
    ramda "0.24.1"
    request "2.88.0"
    request-progress "3.0.0"
    supports-color "5.5.0"
    tmp "0.1.0"
    url "0.11.0"
    yauzl "2.10.0"

it works.

Previously, we used:

cypress@3.6.1:
  version "3.6.1"
  resolved "https://registry.yarnpkg.com/cypress/-/cypress-3.6.1.tgz#4420957923879f60b7a5146ccbf81841a149b653"
  integrity sha512-6n0oqENdz/oQ7EJ6IgESNb2M7Bo/70qX9jSJsAziJTC3kICfEMmJUlrAnP9bn+ut24MlXQST5nRXhUP5nRIx6A==
  dependencies:
    "@cypress/listr-verbose-renderer" "0.4.1"
    "@cypress/xvfb" "1.2.4"
    "@types/sizzle" "2.3.2"
    arch "2.1.1"
    bluebird "3.5.0"
    cachedir "1.3.0"
    chalk "2.4.2"
    check-more-types "2.24.0"
    commander "2.15.1"
    common-tags "1.8.0"
    debug "3.2.6"
    execa "0.10.0"
    executable "4.1.1"
    extract-zip "1.6.7"
    fs-extra "5.0.0"
    getos "3.1.1"
    is-ci "1.2.1"
    is-installed-globally "0.1.0"
    lazy-ass "1.6.0"
    listr "0.12.0"
    lodash "4.17.15"
    log-symbols "2.2.0"
    minimist "1.2.0"
    moment "2.24.0"
    ramda "0.24.1"
    request "2.88.0"
    request-progress "3.0.0"
    supports-color "5.5.0"
    tmp "0.1.0"
    untildify "3.0.3"
    url "0.11.0"
    yauzl "2.10.0"

note: that we're still using 3.6.1 in package.json. And we can confirm amplify is using 3.6.1 when running the test.

nunsie commented 4 years ago

Experiencing the exact same issue, it seems the build step isn't respecting the directory configured in: frontend > artifacts > baseDirectory and caching the entire directory including src and node_modules resulting in a cache size of over 1.2GB, my build folder size should only be 96MB

nunsie commented 4 years ago

removing tests step resolves the issue..

nadinagray commented 4 years ago

Hi -- experiencing the exact same issue. Have tried: removing branches, removing PR previews, creating a new app. @nimacks any suggestions or ideas for a workaround?

Region: us-west-2 App id: d1kf7vuhz1gmur

kahdojay commented 4 years ago

@nadinagerlach we took a look and it appears you are running in a limit of the number of files in your deploy artifacts. This is a known issue for which we're working on a fix. Since it's a per-app limit, one workaround would be to create a new app in the Amplify Console for a new branch. Please give it a try and let us know if it works.

ascorbic commented 4 years ago

@kahdojay Does this mean we can't use PR previews? I'm trying to deploy a moderately-large Gatsby site, which has around 15000 files, and it deployed fine initially but since adding PR previews deploys are failing with this error. Is there a timeline for a fix? If not we'll have to move away from Amplify entirely.