vercel / next.js

The React Framework
https://nextjs.org
MIT License
124.73k stars 26.62k forks source link

`npm run build` sometimes times out building static pages in Next 12 for our repository #35602

Open travis opened 2 years ago

travis commented 2 years ago

Verify canary release

Provide environment information

Operating System:
  Platform: darwin
  Arch: x64
  Version: Darwin Kernel Version 20.6.0: Wed Jan 12 22:22:42 PST 2022; root:xnu-7195.141.19~2/RELEASE_X86_64
Binaries:
  Node: 16.13.1
  npm: 8.1.2
  Yarn: 1.22.4
  pnpm: N/A
Relevant packages:
  next: 12.1.1-canary.18
  react: 17.0.2
  react-dom: 17.0.2

What browser are you using? (if relevant)

No response

How are you deploying your application? (if relevant)

No response

Describe the Bug

Various pages in our app often time out during static page generation when using Next 12. We first noticed this in our Vercel builds, where they time out occasionally, but have recently been able to reproduce this fairly reliably on the command line.

Using node 17 seems to make this worse, but it does happen fairly reliably on 16 as well. Different pages time out in different builds, ie build 1:

info  - Creating an optimized production build
info  - Compiled successfully
info  - Collecting page data
warn  - Restarted static page generation for /u/[handle] because it took more than 60 seconds
warn  - See more info here https://nextjs.org/docs/messages/static-page-generation-timeout
info  - Generating static pages (14/14)
info  - Finalizing page optimization

Page                                       Size     First Load JS
┌ ○ /                                      2.29 kB         754 kB
├   /_app                                  0 B             615 kB
├ ○ /404                                   197 B           615 kB
├ λ /api/fetch-og-tags/[encodedUrl]        0 B             615 kB
├ λ /api/setup-gnome                       0 B             615 kB
├ ○ /login                                 1.45 kB         649 kB
├ ○ /login/[handle]                        485 B           615 kB
├ ○ /logout                                740 B           622 kB
├ ○ /privacy                               750 B           677 kB
├ ○ /register                              3.92 kB         652 kB
├ ○ /settings                              4.94 kB         709 kB
├ ○ /tags/[workspace]/[tag]                2.38 kB         679 kB
├ ○ /tos                                   13.4 kB         689 kB
├ ○ /u/[handle] (1585 ms)                  1.74 kB         748 kB
├ ○ /u/[handle]/[workspace]/[name]         6.71 kB         701 kB
└ ○ /u/[handle]/message                    1.16 kB         616 kB
+ First Load JS shared by all              615 kB
  ├ chunks/framework-7f78491ac389bdeb.js   46.5 kB
  ├ chunks/main-1b52e6399fb0a6ee.js        27.8 kB
  ├ chunks/pages/_app-490938265dfdfcce.js  540 kB
  ├ chunks/webpack-42cdea76c8170223.js     1.07 kB
  └ css/6f3198be8e019a09.css               9.86 kB

build 2:

info  - Creating an optimized production build
info  - Compiled successfully
info  - Collecting page data
warn  - Restarted static page generation for / because it took more than 60 seconds
warn  - See more info here https://nextjs.org/docs/messages/static-page-generation-timeout
warn  - Restarted static page generation for / because it took more than 60 seconds
info  - Generating static pages (14/14)
info  - Finalizing page optimization

Page                                       Size     First Load JS
┌ ○ / (851 ms)                             2.29 kB         754 kB
├   /_app                                  0 B             615 kB
├ ○ /404                                   197 B           615 kB
├ λ /api/fetch-og-tags/[encodedUrl]        0 B             615 kB
├ λ /api/setup-gnome                       0 B             615 kB
├ ○ /login                                 1.45 kB         649 kB
├ ○ /login/[handle]                        485 B           615 kB
├ ○ /logout                                740 B           622 kB
├ ○ /privacy                               750 B           677 kB
├ ○ /register                              3.92 kB         652 kB
├ ○ /settings                              4.94 kB         709 kB
├ ○ /tags/[workspace]/[tag]                2.38 kB         679 kB
├ ○ /tos                                   13.4 kB         689 kB
├ ○ /u/[handle]                            1.74 kB         748 kB
├ ○ /u/[handle]/[workspace]/[name]         6.71 kB         701 kB
└ ○ /u/[handle]/message                    1.16 kB         616 kB
+ First Load JS shared by all              615 kB
  ├ chunks/framework-7f78491ac389bdeb.js   46.5 kB
  ├ chunks/main-1b52e6399fb0a6ee.js        27.8 kB
  ├ chunks/pages/_app-490938265dfdfcce.js  540 kB
  ├ chunks/webpack-42cdea76c8170223.js     1.07 kB
  └ css/6f3198be8e019a09.css               9.86 kB

and sometimes they don't time out at all:

info  - Creating an optimized production build
info  - Compiled successfully
info  - Collecting page data
info  - Generating static pages (14/14)
info  - Finalizing page optimization

Page                                       Size     First Load JS
┌ ○ /                                      2.29 kB         754 kB
├   /_app                                  0 B             615 kB
├ ○ /404                                   197 B           615 kB
├ λ /api/fetch-og-tags/[encodedUrl]        0 B             615 kB
├ λ /api/setup-gnome                       0 B             615 kB
├ ○ /login                                 1.45 kB         649 kB
├ ○ /login/[handle]                        485 B           615 kB
├ ○ /logout                                740 B           622 kB
├ ○ /privacy                               750 B           677 kB
├ ○ /register                              3.92 kB         652 kB
├ ○ /settings                              4.94 kB         709 kB
├ ○ /tags/[workspace]/[tag]                2.38 kB         679 kB
├ ○ /tos                                   13.4 kB         689 kB
├ ○ /u/[handle]                            1.74 kB         748 kB
├ ○ /u/[handle]/[workspace]/[name]         6.71 kB         701 kB
└ ○ /u/[handle]/message                    1.16 kB         616 kB
+ First Load JS shared by all              615 kB
  ├ chunks/framework-7f78491ac389bdeb.js   46.5 kB
  ├ chunks/main-1b52e6399fb0a6ee.js        27.8 kB
  ├ chunks/pages/_app-490938265dfdfcce.js  540 kB
  ├ chunks/webpack-42cdea76c8170223.js     1.07 kB
  └ css/6f3198be8e019a09.css               9.86 kB

λ  (Server)  server-side renders at runtime (uses getInitialProps or getServerSideProps)
○  (Static)  automatically rendered as static HTML (uses no initial props)

We've tried to verify that there aren't network requests happening, and indeed this seems to happen even if we strip the _app_ and _document files out.

As far as we can tell this starts happening when we include some of our dependencies in a file, and is not dependent on any particular component/application structure.

Expected Behavior

We expect pages to take roughly the same amount of time to build across different runs as they did in Next 11.

Downgrading to Next 11.1 does seem to fix this, but strangely enough adding a .babelrc to disable SWC builds does not seem to.

To Reproduce

git clone https://github.com/mysilio-co/garden.git
cd garden
git checkout nextjs-bugreport
npm install
npm run build

You may need to run npm run build more than once because unfortunately the results seem to be somewhat nondeterministic - the first time I ran this it completed with:

info  - Creating an optimized production build
info  - Compiled successfully
info  - Collecting page data
info  - Generating static pages (14/14)
info  - Finalizing page optimization

Page                                       Size     First Load JS
┌ ○ /                                      2.28 kB         753 kB
├   /_app                                  0 B             615 kB
├ ○ /404                                   196 B           615 kB
├ λ /api/fetch-og-tags/[encodedUrl]        0 B             615 kB
├ λ /api/setup-gnome                       0 B             615 kB
├ ○ /login                                 1.44 kB         649 kB
├ ○ /login/[handle]                        483 B           615 kB
├ ○ /logout                                737 B           622 kB
├ ○ /privacy                               749 B           676 kB
├ ○ /register                              3.92 kB         652 kB
├ ○ /settings                              4.94 kB         708 kB
├ ○ /tags/[workspace]/[tag]                2.37 kB         678 kB
├ ○ /tos                                   13.4 kB         689 kB
├ ○ /u/[handle]                            1.73 kB         747 kB
├ ○ /u/[handle]/[workspace]/[name]         6.71 kB         701 kB
└ ○ /u/[handle]/message                    1.16 kB         616 kB
+ First Load JS shared by all              615 kB
  ├ chunks/framework-7f78491ac389bdeb.js   46.5 kB
  ├ chunks/main-914fbfab4f90b52f.js        27.7 kB
  ├ chunks/pages/_app-33bda76b787f2992.js  540 kB
  ├ chunks/webpack-42cdea76c8170223.js     1.07 kB
  └ css/bc22ef3cff942bbd.css               9.83 kB

but the second time it did this:

info  - Creating an optimized production build
info  - Compiled successfully
warn  - Restarted collecting page data for / because it took more than 60 seconds
warn  - See more info here https://nextjs.org/docs/messages/static-page-generation-timeout
warn  - Restarted collecting page data for /settings because it took more than 60 seconds
warn  - Restarted collecting page data for /u/[handle] because it took more than 60 seconds
info  - Collecting page data
info  - Generating static pages (14/14)
info  - Finalizing page optimization

Page                                       Size     First Load JS
┌ ○ /                                      2.28 kB         753 kB
├   /_app                                  0 B             615 kB
├ ○ /404                                   196 B           615 kB
├ λ /api/fetch-og-tags/[encodedUrl]        0 B             615 kB
├ λ /api/setup-gnome                       0 B             615 kB
├ ○ /login (1042 ms)                       1.44 kB         649 kB
├ ○ /login/[handle] (967 ms)               483 B           615 kB
├ ○ /logout (935 ms)                       737 B           622 kB
├ ○ /privacy                               749 B           676 kB
├ ○ /register                              3.92 kB         652 kB
├ ○ /settings                              4.94 kB         708 kB
├ ○ /tags/[workspace]/[tag]                2.37 kB         678 kB
├ ○ /tos                                   13.4 kB         689 kB
├ ○ /u/[handle]                            1.73 kB         747 kB
├ ○ /u/[handle]/[workspace]/[name]         6.71 kB         701 kB
└ ○ /u/[handle]/message (989 ms)           1.16 kB         616 kB
+ First Load JS shared by all              615 kB
  ├ chunks/framework-7f78491ac389bdeb.js   46.5 kB
  ├ chunks/main-914fbfab4f90b52f.js        27.7 kB
  ├ chunks/pages/_app-33bda76b787f2992.js  540 kB
  ├ chunks/webpack-42cdea76c8170223.js     1.07 kB
  └ css/bc22ef3cff942bbd.css               9.83 kB
balazsorban44 commented 2 years ago

There should be a .next/trace file after the build. Could you provide it for the different runs (when there is and there is no issue)?

This should help us investigate what's taking up the time.

travis commented 2 years ago

ok here's one that failed completely:

info  - Creating an optimized production build
info  - Compiled successfully
warn  - Restarted collecting page data for / because it took more than 60 seconds
warn  - See more info here https://nextjs.org/docs/messages/static-page-generation-timeout

> Build error occurred
Error: Collecting page data for / is still timing out after 2 attempts. See more info here https://nextjs.org/docs/messages/page-data-collection-timeout

trace.txt

had to add .txt to the filename to get it to upload, fwiw

travis commented 2 years ago

here's another run:

info  - Creating an optimized production build
info  - Compiled successfully
info  - Collecting page data
warn  - Restarted static page generation for /tos because it took more than 60 seconds
warn  - See more info here https://nextjs.org/docs/messages/static-page-generation-timeout
warn  - Restarted static page generation for /u/[handle] because it took more than 60 seconds
warn  - Restarted static page generation for /u/[handle] because it took more than 60 seconds

> Build error occurred
Error: Static page generation for /u/[handle] is still timing out after 3 attempts. See more info here https://nextjs.org/docs/messages/static-page-generation-timeout

trace.txt

travis commented 2 years ago

ok and after 5-6 tries, here's a successful run!

info  - Creating an optimized production build
info  - Compiled successfully
info  - Collecting page data
info  - Generating static pages (14/14)
info  - Finalizing page optimization

Page                                       Size     First Load JS
┌ ○ /                                      2.28 kB         753 kB
├   /_app                                  0 B             615 kB
├ ○ /404                                   196 B           615 kB
├ λ /api/fetch-og-tags/[encodedUrl]        0 B             615 kB
├ λ /api/setup-gnome                       0 B             615 kB
├ ○ /login                                 1.44 kB         649 kB
├ ○ /login/[handle]                        483 B           615 kB
├ ○ /logout                                737 B           622 kB
├ ○ /privacy                               749 B           676 kB
├ ○ /register                              3.92 kB         652 kB
├ ○ /settings                              4.94 kB         708 kB
├ ○ /tags/[workspace]/[tag]                2.37 kB         678 kB
├ ○ /tos                                   13.4 kB         689 kB
├ ○ /u/[handle]                            1.73 kB         747 kB
├ ○ /u/[handle]/[workspace]/[name]         6.71 kB         701 kB
└ ○ /u/[handle]/message                    1.16 kB         616 kB
+ First Load JS shared by all              615 kB
  ├ chunks/framework-7f78491ac389bdeb.js   46.5 kB
  ├ chunks/main-914fbfab4f90b52f.js        27.7 kB
  ├ chunks/pages/_app-69500b13a3e692d4.js  540 kB
  ├ chunks/webpack-42cdea76c8170223.js     1.07 kB
  └ css/bc22ef3cff942bbd.css               9.83 kB

trace.txt

balazsorban44 commented 2 years ago

OK, I can reproduce locally.

On first look, the trace files could not tell me where the problem is, but I'll try to discuss with the others if they have any ideas.

A few remarks:

  1. Running your repro locally in development shows the following warning:

    It looks like there are several instances of 'styled-components' initialized in this application. This may cause dynamic styles to not render properly, errors during the rehydration process, a missing theme prop, and makes your application bigger without good reason.

It might be worth looking into, as the initial load seems to be 600+ kB, minifed, which sounds unnecessarily huge.

  1. I cannot confirm how, but I think the used dependencies are playing an important role here. I am not familier with the packages so I'll keep digging, but it would be nice to try to minimize the reproduction, removing any unnecessary dependencies/piece of code that does not contribute to the issue, so it's easier to investigate. Could you try to do that?
travis commented 2 years ago

Thanks @balazsorban44 ! I'll take a look at de-duping or completely removing styled-components and will see if I can get this to go away by stripping out any unused dependencies - appreciate you!

travis commented 2 years ago

Ok @balazsorban44 I've managed to narrow down the issue - not terribly surprisingly the problem appears to be our custom swrlit dependency, but since that isn't a very big repo I suspect the issue might be in one of its dependencies.

I've published a minimum-viable reproduction here:

https://github.com/mysilio-co/garden/tree/minimum-reproducible-nextjs-bug

It builds pretty cleanly by default, but if you head into _app.jsx and uncomment swrlit the build may time out (it does very reliably for me locally). If you re-comment that and then uncomment the @inrupt/solid-client-authn-browser dependency you'll see every page jump by 100KB, but builds don't appear to fully time out the way they do when we import swrlit.

I'm going to keep debugging this as I have time, but if you have any ideas or suggestions on what might be wrong with these packages please let me know!

travis commented 2 years ago

for what it's worth, this PR seems to have helped a lot:

https://github.com/mysilio-co/garden/pull/118

as you can see above we're investigating the heavy dependencies with the Inrupt team.

Would be great for Next.js to handle situations like this with something other than segmentation faults, so this bug report still feels open, but the root cause at least seems clearer at this point.

Chukwu3meka commented 1 year ago

// next.config.js

module.exports = {
  // time in seconds of no pages generating during static
  // generation before timing out
  staticPageGenerationTimeout: 1000,
}

The key is to increase timeout to collect page data