Cold start taking 10-12 seconds to render static page

[mr-rpl]

Before opening, please confirm:

• [X] I have checked to see if my question is addressed in the FAQ.
• [X] I have searched for duplicate or closed issues.
• [X] I have read the guide for submitting bug reports.
• [X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
• [X] I have removed any sensitive information from my code snippets and submission.

App Id

d15y9mlar87m44

AWS Region

us-east-1

Amplify Hosting feature

Not Applicable

Describe the bug

Upon new deployment and/or after the app goes idle, we are experiencing a 10-12 second delay on any page of our Next13 application. We are on the new Web Compute platform.

Mere speculation wants to blame Lambda Cold Start -- but I am not 100% sure if that is the case -- either way, a public facing UI should never be faced with a 10-12 second cold start.

Expected behavior

Loads in reasonable time

Reproduction steps

1. Fresh deploy and visit the site
2. Let sit idle for a period of time and visit the site

Build Settings

No response

Log output

No response

Additional information

We have a sample build up here

Screenshot shows a 10 second request

[soplan]

that's funny. Vercel also has cold start issues. Not sure why you believe this is only applicable to AWS. Vercel uses AWS for hosting.

Either way, use route 53 health checks and gone are your cold start issues. At least for us it is.

[kachkaev]

@soplan how often do you run those checks? Once in how many minutes?

PS: I experienced similar issues on Vercel before, it was to do with heavy lambda functions taking time to warm up: https://github.com/orgs/vercel/discussions/496

As a workaround, we did this:

• .github/workflows/warm-up-vercel.yml
• .github/actions/warm-up-vercel/action.yml

I’d like to do something similar against an Amplify project, so I wonder what frequency is enough. If an Amplify app is kept warm all the time, how does this affect billing?

[vafPER]

Any update on this? @Jay2113

[afern247]

This is simply unacceptable... 😡 What is the status on this?? This PR has been open since Dec 21, 2022!

@josefaidt I know you always help on the iOS side and others, is there a way to escalate this? This is unusable at prod level and very frustrating... cc: @hloriii

[swaminator]

@colin-chadwick @afern247 can you share your appIDs please?

[afern247]

@colin-chadwick @afern247 can you share your appIDs please?

It's d3fooq4aiahxc0

[colin-chadwick]

Mine is d16xb97oi6wxj4

[michrome]

Thank you for your patience here. We have recently deployed improvements to the startup time for Next.js 12 and 13 that have larger numbers of dependencies. The improvements are automatically applied during branch builds.

We are working on improving the performance of apps with large/many static assets and apps that use next/image. We will deploy these improvements in the coming months.

[michrome]

Hey @colin-chadwick you might want to revise your cache-control header. Right now I see it's returning private, no-cache, no-store, max-age=0, must-revalidate, missing opportunities to cache at the client, CDN and app server.

[colin-chadwick]

Hey @michrome, thanks for letting me know! Setting up a health check that regularly visits the page to keep it warm definitely helped as well. Do you have any guides to setup these headers? I deployed the site on Amplify without making any further adjustments. It seems like there are still some performance options I’m missing out on.

[soplan]

@soplan how often do you run those checks? Once in how many minutes?

PS: I experienced similar issues on Vercel before, it was to do with heavy lambda functions taking time to warm up: https://github.com/orgs/vercel/discussions/496

As a workaround, we did this:

• .github/workflows/warm-up-vercel.yml
• .github/actions/warm-up-vercel/action.yml

I’d like to do something similar against an Amplify project, so I wonder what frequency is enough. If an Amplify app is kept warm all the time, how does this affect billing?

you only get 2 options when configuring health checks in route 53. Just keep it default. I believe it's every 30 sec or 10 sec?

Either way, it affects your billing by 1 USD per month as each health check costs you 1 USD. it's a bargain.

[afern247]

Thank you for your patience here. We have recently deployed improvements to the startup time for Next.js 12 and 13 that have larger numbers of dependencies. The improvements are automatically applied during branch builds.

We are working on improving the performance of apps with large/many static assets and apps that use next/image. We will deploy these improvements in the coming months.

It seemed to have improved, not the best performance but at least is not taking 6 seconds now... Until it's fixed I'm fine with it under 1 sec. Thank you.

Note, I'm still getting this error message (on Firefox only) from time to time:

[ekaterina4952]

Hey @colin-chadwick you might want to revise your cache-control header. Right now I see it's returning private, no-cache, no-store, max-age=0, must-revalidate, missing opportunities to cache at the client, CDN and app server.

I am curious what cache setting will help when initial call happens.

[abepuentes]

To the people using route 58 as a warmup option, as far as I understand that will keep up just one lambda warm up, so when you have a couple of concurrent connection the new users will still experience cold start issues. SST let you set up a couple of warm functions to resolve this, if someone wanna try today until this is completely resolve here. Btw has anyone experiencing higher than expected building cost from amplify hosting?

[punit1108]

Need to know if someone is working on this or not? @hloriii @Jay2113

Setting up Health Check eats up a whole lot of Amplify Bandwidth (as the home page of my website is around 3-4 MBs), resulting in higher bills. If no one is working on this, we would be better off finding some permanent solution to this.

[marcantoineveilleux]

@punit1108 on my side, I've created a very simple page that calls a dummy SSR call to make sure the health check spans a machine and not hit the next cache. This is what the health check is pinging. Works well.

[punit1108]

@marcantoineveilleux Oh, good to hear that. Very curious to know how did you manage to do that. Any resource where I can read this up?

[marcantoineveilleux]

@punit1108

This isn't exactly what I did, but should work: https://stackoverflow.com/questions/57956476/how-to-set-up-an-endpoint-for-health-check-on-next-js

[punit1108]

@marcantoineveilleux This worked beautifully. Thanks!

Although it's a great workaround, it still costs money for no reason, it would really be great if amplify team fixes this issue asap.

[jbreemhaar]

Might not work for everyone, but in a Nextjs app (pages router) I've moved Amplify.configure to async clientside. My TTFB dropped immediately as you can see in the screenshot.

Most of our pages is pure marketing, stuff which needs Amplify (Auth, API (e.g. user data/orders), Maps) gets lazyloaded.

[wheresthecode]

Adding a health check brought my load time from about 10 seconds to less than a second. I'm just going to stick with that. Hopefully it isn't too expensive :)

[calavera]

Hey everyone 👋

We just released some changes to improve how long it takes to serve content from Next.js apps. These changes require a new deploy in your application to take effect. The expected outcome is that your sites should load quicker than before.

If you have comments or questions don't hesitate to reach us via AWS support.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.

[github-actions[bot]]

This issue has been automatically locked.