Closed erawk closed 3 months ago
Hi, given the reproduction, I am unable to reproduce the issue. Here are the numbers I see upon deploying:
Route (pages) Size First Load JS
--
01:59:01.719 | ┌ ○ / 274 B 78.7 kB
01:59:01.719 | └ ○ /404 212 B 78.6 kB
01:59:01.719 | + First Load JS shared by all 78.4 kB
01:59:01.720 | ├ chunks/framework-3911a61406e859ea.js 45.5 kB
01:59:01.720 | ├ chunks/main-961c23d3ed15b7d4.js 31.7 kB
01:59:01.720 | ├ chunks/pages/_app-ef9da0a6572b3989.js 243 B
01:59:01.720 | └ chunks/webpack-adbad1f85bb5cebc.js 962 B
Can you make sure the reproduction is complete? libphonenumber-js
is also not present for example.
Build a serverless function page
Are you referring to an API route? Could you add one that seems to fail the build?
I was seeing similar results on 13.0.2
Traced Next.js server files in: 18.254s
--
01:41:37.433 | Warning: Max serverless function size of 50 MB compressed or 250 MB uncompressed reached
01:41:37.433 | Serverless Function's page: api/auth/[...nextauth].js
01:41:37.450 | Large Dependencies Uncompressed size Compressed size
01:41:37.450 | node_modules/.pnpm/@prisma+client@4.5.0_prisma@4.5.0 45.7 MB 15.8 MB
01:41:37.450 | node_modules/.pnpm/@swc+core-linux-x64-gnu@1.3.10 43.2 MB 14.4 MB
01:41:37.451 | node_modules/.pnpm/@swc+core-linux-x64-musl@1.3.10 42.7 MB 14.2 MB
01:41:37.451 | node_modules/.pnpm/esbuild-linux-64@0.15.13 8.58 MB 3.6 MB
01:41:37.451 | node_modules/.pnpm/next@13.0.2_biqbaboplfbrettd7655fr4n2y 12.2 MB 3.12 MB
01:41:37.451 | node_modules/.pnpm/webpack@5.74.0_@swc+core@1.3.10 3.93 MB 953 kB
01:41:37.451 | apps/app/.next 4.71 MB 825 kB
01:41:37.451 | node_modules/.pnpm/react-dom@18.2.0_react@18.2.0 1.72 MB 415 kB
01:41:37.451 | node_modules/.pnpm/@mantine+core@5.7.1_etrvyeh76ry4n4cxjc37hheh7y 908 kB 321 kB
01:41:37.453 | node_modules/.pnpm/@mantine+core@5.7.1_rpbu45oak5njhv4mrqfytqjmse 908 kB 321 kB
01:41:37.454 | node_modules/.pnpm/caniuse-lite@1.0.30001429 844 kB 312 kB
01:41:37.454 | node_modules/.pnpm/uglify-js@3.17.4 1.18 MB 213 kB
01:41:37.454 | node_modules/.pnpm/terser@5.15.1 962 kB 185 kB
01:41:37.454 |
01:41:37.454 | All dependencies 55 MB 56.3 MB
01:41:37.458 | Serverless Function's page: api/trpc/[trpc].js
01:41:37.467 | Large Dependencies Uncompressed size Compressed size
01:41:37.467 | node_modules/.pnpm/@prisma+client@4.5.0_prisma@4.5.0 45.7 MB 15.8 MB
01:41:37.467 | node_modules/.pnpm/@swc+core-linux-x64-gnu@1.3.10 43.2 MB 14.4 MB
01:41:37.467 | node_modules/.pnpm/@swc+core-linux-x64-musl@1.3.10 42.7 MB 14.2 MB
01:41:37.467 | node_modules/.pnpm/esbuild-linux-64@0.15.13 8.58 MB 3.6 MB
01:41:37.467 | node_modules/.pnpm/next@13.0.2_biqbaboplfbrettd7655fr4n2y 12.2 MB 3.12 MB
01:41:37.468 | node_modules/.pnpm/webpack@5.74.0_@swc+core@1.3.10 3.93 MB 953 kB
01:41:37.468 | apps/app/.next 4.71 MB 825 kB
01:41:37.468 | node_modules/.pnpm/react-dom@18.2.0_react@18.2.0 1.72 MB 415 kB
01:41:37.468 | node_modules/.pnpm/@mantine+core@5.7.1_etrvyeh76ry4n4cxjc37hheh7y 908 kB 321 kB
01:41:37.468 | node_modules/.pnpm/@mantine+core@5.7.1_rpbu45oak5njhv4mrqfytqjmse 908 kB 321 kB
01:41:37.468 | node_modules/.pnpm/caniuse-lite@1.0.30001429 844 kB 312 kB
01:41:37.468 | node_modules/.pnpm/uglify-js@3.17.4 1.18 MB 213 kB
01:41:37.469 | node_modules/.pnpm/terser@5.15.1 962 kB 185 kB
01:41:37.469 |
01:41:37.469 | All dependencies 55.3 MB 56.3 MB
01:41:37.469 | Max serverless function size was exceeded for 2 functions
01:41:37.479 | Created all serverless functions in: 10.230s
01:41:37.490 | Collected static files (public/, static/, .next/static): 7.761ms
01:41:43.350 | Build Completed in /vercel/output [3m]
01:41:55.987 | Failed to process build result for "api/auth/[...nextauth]". Data: {"type":"Lambda"}.
01:41:55.989 | Error: The Serverless Function "api/auth/[...nextauth]" is 55.12mb which exceeds the maximum size limit of 50mb. Learn More: https://vercel.link/serverless-function-size
Rolling back to 13.0.1 fixed the builds.
@balazsorban44 Thanks for the reply! Indeed, the problem is with API routes, as @JoeKarow also notes.
Rolling back to 13.0.1 solved it for us as well. Notice that @JoeKarow is also using Prisma.
There's a bunch more infrastructure to set-up to reproduce the bug, but I think there's two strong cases so far that there's an issue here.
Thanks, I'm having a look at the second reproduction.
It's a monorepo, so I'm currently trying to cut down to the core minimal/reproduction. (As a side note, providing a minimal but complete reproduction while opening an issue is always preferable for a quicker resolution :green_heart:)
Hey @balazsorban44 @JoeKarow
I've tried to update a barebones repro with some of the signals between my private repo where this is happening and @JoeKarow 's public repo. I can't seem to get a repro with my attempts thus far:
Common packages:
prisma
@next/bundle-analyzer
superjson
In my repro app, I have an API route, and both an SSR and an SSG route that invokes Prisma. I still can't repro it.
https://github.com/erawk/next-app-bug-report-serverless-functions-payload-size
The current fix in our app so that we can run on 13.0.2
is to add an entry in package.json
with a postbuild
script:
"postbuild": "sh bin/postbuild.sh",
That has a stanza that looks like
# Make the large files zero bytes for the `musl` SWC
echo "Nuking unnecessary @swc/core files in node_modules/@swc/core-linux-x64-musl"
DISK_USAGE_MUSL_BEFORE=`du -h node_modules/@swc/core-linux-x64-musl/*`
echo "${DISK_USAGE_MUSL_BEFORE}"
echo '' > node_modules/@swc/core-linux-x64-musl/swc
echo '' > node_modules/@swc/core-linux-x64-musl/swc.linux-x64-musl.node
DISK_USAGE_MUSL_AFTER=`du -h node_modules/@swc/core-linux-x64-musl/*`
echo "${DISK_USAGE_MUSL_AFTER}"
This effectively wipes out the unnecessary SWC musl
libc that we don't need.
This is super hacky and really has a ton of unintended consequences if we actually do end up needing musl
libc in @swc/core
.
My guess two fold:
gnu
and musl
@swc/core
modules has always been there, as per this SWC issueThe combination of both has pushed this beyond the allowable 50MB threshold.
I've confirmed in the repro repo that both the -gnu
and -musl
versions of @swc/core
are present via the postbuild.sh
script:
...
What is the state of @swc/core modules in node_modules/@swc/core-linux-x64-*
--
10:24:28.344 | 76M node_modules/@swc/core-linux-x64-gnu
10:24:28.345 | 76M node_modules/@swc/core-linux-x64-musl
Rolling back to 13.0.1 fixed the builds.
Hey! Can you share a list of asset/dependency sizes when deploying 13.0.1? Then we can tell which part actually became larger in the newer version.
@shuding I'm not sure how I can find out that info. I'm deploying to Vercel and a successful build doesn't show the size of the bundled lambdas.
Here are two Vercel full build logs from today from some preview branches, one successful build on 13.0.1 and a failed build on 13.0.2.
@JoeKarow
I'm not sure how I can find out that info. I'm deploying to Vercel and a successful build doesn't show the size of the bundled lambdas.
Add a NEXT_DEBUG_FUNCTION_SIZE=1
in vercel dashboard ?
@belgattitude Ahh, thanks! 13.0.3 looks like it fixed the size issue. It's still including both of the SWC packages - are they supposed to be bundled?
Yep it's weird I'm trying to reproduce here: https://github.com/belgattitude/compare-package-managers. But I don't use the experimental corepack (so I guess I have a lower version of pnpm)
Can you try a pnpm why -r @swc/core
? (you shouldn't have any)
Sure, here you are! It looks like only our Storybook repo is using @swc/core
. @weareinreach/storybook
is a standalone package - it's not a dependency to anything.
Would move storybook-addon-next is in your devDependencies as a first step... and if it's already done, I guess there's a bug in the outfileFileTracing / nft.
Edit: Seeing terser and uglify, it might be actually related to nft (?) Why dev deps here ? They should be dropped (maybe @balazsorban44 can confirm). Just tested a standalone build to see locally, maybe a clue:
PS: about having musl and glibc packages is another problem. AFAIK the only pm that is able to not install only what's needed is yarn 3+ (https://yarnpkg.com/configuration/yarnrc#supportedArchitectures). But I agree @swc/core should be dropped, so I shouldn't make any difference
Moving storybook-addon-next
to devDependencies did not change the size.
Mmm, by chance do you have some storybook in deps ? Niot sure what you're testingn but I see this https://github.com/weareinreach/InReach/blob/a82a69851e284c71f3cca7011a6813b9b8e09467/packages/storybook/package.json#L18). Would it fix the issue ?
I pulled out the vite builder and there was no change. The only time anything storybook-related is referenced outside of the @weareinreach/storybook
package is just an eslint
plugin in a shared config package.
@erawk
What does your function bundle look like when building on 13.0.3
?
Hey! Can you share a list of asset/dependency sizes when deploying 13.0.1? Then we can tell which part actually became larger in the newer version.
Here you go, @shuding . TLDR -- in versions 13.0.2
+, am now seeing a @swc/core
included.
Surprisingly, the @swc/core
double include for both musl
and gnu
has disappeared. Not sure how this happened.
Each of the results above are from an uncached build.
Now to track down why @swc/core
is included...
Thanks @belgattitude for the tip on pnpm why -r @swc/core
.
@erawk Are you still using that post-build script that wipes @swc/core-linux-x64-musl
?
@JoeKarow no, I've removed it. Though maybe the results were cached between builds.
Let me re-run without a deploy cache.
@JoeKarow 🤦 Yes, it was nuked thanks to the build cache. An uncached build exceed the threshold:
Here's the output from npm explain @swc/core
. Doesn't seem to be anything that should be pulling it in that's not dev
Okay - I spoke too soon... 13.0.3 has bloated back up again.
Hi, any chance someone who has a repo that consistently fails with this above v13.0.1
could try the patch from https://github.com/vercel/next.js/pull/43069 via yarn add next@https://files-99l7vkuaq-ijjk-testing.vercel.app
and confirming whether this alleviates the problem or not?
Hi, any chance someone who has a repo that consistently fails with this above
v13.0.1
could try the patch from #43069 viayarn add next@https://files-99l7vkuaq-ijjk-testing.vercel.app
and confirming whether this alleviates the problem or not?
I tried it, but it doesn't work for me. Had to downgrade to 13.0.1
to get my app to build.
Hi, any chance someone who has a repo that consistently fails with this above
v13.0.1
could try the patch from #43069 viayarn add next@https://files-99l7vkuaq-ijjk-testing.vercel.app
and confirming whether this alleviates the problem or not?
Hey @ijjk - sorry, no luck. There's no change in the build sizes
Large Dependencies Uncompressed size Compressed size -- node_modules/.prisma/client 44.9 MB 15.6 MB node_modules/@swc/core-linux-x64-gnu 43.1 MB 14.4 MB node_modules/@swc/core-linux-x64-musl 42.7 MB 14.3 MB node_modules/esbuild-linux-64/bin 8.36 MB 3.52 MB node_modules/next/dist 8.25 MB 2.2 MB node_modules/@fortawesome/pro-regular-svg-icons 3.55 MB 1.09 MB node_modules/webpack/lib 3.35 MB 861 kB node_modules/react-dom/cjs 1.72 MB 414 kB node_modules/@prisma/client 1.23 MB 267 kB node_modules/terser/dist 959 kB 183 kB .next/server/chunks 673 kB 143 kB node_modules/libphonenumber-js/build 355 kB 118 kB pages/t/[slug].js 725 kB 109 kB node_modules/libphonenumber-js/es6 326 kB 108 kB node_modules/instantsearch.js/cjs 352 kB 102 kB node_modules/webpack/schemas 565 kB 88.5 kB modules/directoryTopics/constants 517 kB 77.3 kB All dependencies 170 MB 55.9 MB
@erawk can you share the specific commit you deployed to get that result, I'm not able to reproduce the size issue on the latest commit here https://github.com/erawk/next-app-bug-report-serverless-functions-payload-size/commit/2c95975ac099fd027a4cafdeeb9ae2b996fca2b4
Hi, we have landed a new experimental.outputFileTracingIgnores
config that allows adding custom globs to ignore from the traces which can allow making the function size more controlled when certain dependencies are known to not be needed at runtime.
Example:
module.exports = {
experimental: {
outputFileTracingIgnores: ['**swc+core**', '**esbuild**', ']
}
}
It seems usage of storybook is causing swc to be traced due to the way it is configured.
we have landed a new
experimental.outputFileTracingIgnores
config that allows adding custom globs to ignore from the traces which can allow making the function size more controlled when certain dependencies are known to not be needed at runtime.
Yes! This definitely solved our problem. We added the fragment to exclude swc/core
and our payload sizes are no longer overflowing:
Here's the relevant section from our next.config.js
for others:
module.exports = {
experimental: {
esmExternals: true,
outputFileTracingIgnores: ['**swc/core**'], // See https://github.com/vercel/next.js/issues/42641#issuecomment-1320713368
swcPlugins: [['next-superjson-plugin', {}]], // Use SuperJSON in getStaticProps via SWC
},
...
}
It seems usage of storybook is causing swc to be traced due to the way it is configured.
Interesting! We don't use Storybook. We do have ladle. There's probably something else that's pulled @swc/core
in.
I am still curious what changed between 13.0.1
and 13.0.2
that made @swc/core
be included.
@erawk If you can share two commits with only bumping 13.0.1
and 13.0.2
showing this being the only change causing the increase would be happy to investigate this further! The reproduction we did see with this was showing this on 13.0.1
as well though and haven't seen this occurring only after updating 13.0.2
.
Bumping to 13.0.5-canary.2
fixed the bloat with experimental.outputFileTracingIgnores
.
I am experiencing a different issue now - my serverless functions aren't able to use the Prisma client. It doesn't get bundled during the Vercel deploy, but oddly enough, it does if I do vercel build
using the cli locally.
Is there a docker container which replicates Vercel's server build/deploy process so I can run it locally when troubleshooting/testing?
@erawk If you can share two commits with only bumping
13.0.1
and13.0.2
showing this being the only change causing the increase would be happy to investigate this further! The reproduction we did see with this was showing this on13.0.1
as well though and haven't seen this occurring only after updating13.0.2
.
Hi, just wan't to inform that my company is also experiencing the exact problem when upgrading to 13.0.2 from 13.0.1. I tried to make a reproduce repo but the problem isn't showing.
Here's one of the example on regular next-auth route with prisma and siwe dependencies.
Let me know what can I provide to help.
Upgraded to 13.0.5
, unfortunately still causing issue. I see @JoeKarow referenced this and said 13.0.5-canary.2 fixed it, but the stable version does not (using npm, standard t3 stack).
Hey @liamross - what fixed it for us was to ensure we have the outputFileTracingIgnores
set-up, as in my comment here
module.exports = {
experimental: {
esmExternals: true,
outputFileTracingIgnores: ['**swc/core**'], // See https://github.com/vercel/next.js/issues/42641#issuecomment-1320713368
swcPlugins: [['next-superjson-plugin', {}]], // Use SuperJSON in getStaticProps via SWC
},
...
}
It wasn't just updating to 13.0.5
that solved it, but also ensuring we add the outputFileTracingIgnores
block in our next.config.js
.
That solved the build issues, however now the serverless functions crash and I can't access the site. Here is my function error output if anyone else is having the same problem (for now the fix on my end seems to be simply downgrading to 13.0.1
).
[GET] /
09:46:31:89
2022-11-25T17:46:32.077Z undefined ERROR Cannot find module './initialize-require-hook'
Require stack:
- /var/task/apps/app/node_modules/next/dist/server/next-server.js
- /var/task/apps/app/___next_launcher.cjs
2022-11-25T17:46:32.077Z undefined ERROR Did you forget to add it to "dependencies" in `package.json`?
RequestId: a0e959c0-0f5e-427d-8e38-d421ee6cecec Error: Runtime exited with error: exit status 1
Runtime.ExitError
@liamross one thing you can try is:
13.0.5
next.config.ts
for outputFileTracingIgnores
post-build.sh
step in your deploy process to zero-byte one of the @swc/core-linux
librariesAn example of post-build.sh
is what I posted earlier in : https://github.com/vercel/next.js/issues/42641#issuecomment-1310422980
This provides the @swc/core
binary for compilation at run time, but not two of them. For Vercel, I was OK with zeroing out the musl
version. But if you are using say Alpine Linux, you will have to keep the musl
one and zero out the other one.
I don't really recommend this, as it could lead to unexpected consequences if your libc changes from underneath you, but it might help determine if you do need @swc/core
for your deploys.
I'm in a branch where I'm trying to upgrade dependencies and I'm using @swc/core
and @swc/jest
in a component library's Jest unit tests. My Next.js app is v^12 and I'm seeing the same issue with every distribution of swc attempting to be loaded to a serverless function.
I don't have experimental.outputFileTracingIgnores
in that version of Next.js
I'm not sure if this is better served in the SWC repo, but here's an example diff in lockfile:
What should I do?
This is still broken on 13.0.6
unfortunately
For me, this bug appeared when I had @swc/core
listed as a dependency in my package.json—no matter whether or not it was used by my serverless functions, @swc/core
was included in the bundle for every single serverless function, and added up to close to the 50MB limit.
I ran npm un @swc/core
and redeployed my project with no other code changes, and the issue was resolved.
For me, this bug appeared when I had
@swc/core
listed as a dependency in my package.json—no matter whether or not it was used by my serverless functions,@swc/core
was included in the bundle for every single serverless function, and added up to close to the 50MB limit.I ran
npm un @swc/core
and redeployed my project with no other code changes, and the issue was resolved.
Unfortunately this did not fix it for me 😞
Got it working with basically the steps everyone has been saying, not sure why it was dying before but now the basic steps seem to work. Here's a summary:
NOTE: still seeing approx 20 MB increase in uncompressed sizes across my serverless functions, this update is still bad and bundling unnecessary stuff, but at least I'm taking care of the big offenders to get it back below the cutoff
NEXT_DEBUG_FUNCTION_SIZE=1
to Environment Variables, deploy 13.0.1
13.0.1
to 13.0.6
13.0.6
which fails, but tells you the largest files13.0.1
deployment to determine which are unnecessary/your_app/next.config.mjs
or wherever it livesconfig.experimental.outputFileTracingIgnores
**swc/core**
type things others have done, that works tooAdd the following lines or whatever yours look like:
const nextConfig = {
+ experimental: {
+ outputFileTracingIgnores: [
+ 'node_modules/@swc/core-linux-x64-gnu',
+ 'node_modules/@swc/core-linux-x64-musl',
+ 'node_modules/esbuild-linux-64/bin', // doesn't work, can ignore
+ 'node_modules/webpack/lib', // doesn't work, can ignore
+ ],
+ },
};
Edit: I'm noticing that node_modules/esbuild-linux-64/bin
and node_modules/webpack/lib
are still being included which is accounting for 12 / 20 of the unaccounted for MB, if I can get those to be correctly excluded it will be close enough to the normal size that I'll be tentatively happy
I just tried the above 2 things after 13.0.7
was released and it seems to have fixed it for me. Not sure if anything changed or I was just doing something wrong before, but at least for me it's "fixed" 😄
I have removed the experimental section and my builds are working again, I think after 13.0.7 it is fixed (now on 13.1.0) 🥳
I'm seeing this same behavior after upgrading to "13.1.1"
within an NX monorepo. I was able to work around this by removing the @swc/core
dependency.
Just to add that this also happens when using the next-federation-plugin.
Related issue: https://github.com/module-federation/universe/issues/561
Having the same issue after upgrading packages from my project (https://github.com/baptisteArno/typebot.io/commit/d2880cdf2dc44cd370d975c5dffb63121759d776)
I can see tons of unnecessary dev dependencies here: swc, esbuild, rollup, webpack, terser, uglify 🤔
After running pnpm why -r @swc/core
I figured it is @ladel/react
that is using @vitejs/plugin-react-swc
that uses @swc/core
I first tried to move @ladel/react
to dev dependencies, but it didn't work, so I downgraded @ladel/react
from 2.9.0 to 2.5.1
outputFileTracingIgnores
has been moved toexperimental.outputFileTracingExcludes
. Please update your next.config.js file accordingly.
We have just upgraded to nextjs 13.2.4 and now we are having the same problem 😢
It's a monorepo where we have packages/ and apps/ folders. We have @swc/core installed in the root and it's used to compile our frontend libraries inside the packages/ folder.
And when deploying to production (using vercel deploy --prod
) the total code uploaded is 1.2gb 😨
I have noticed that it only happens in our ci (linux), in local (macos) the problem is not present
We have just upgraded to nextjs 13.2.4 and now we are having the same problem 😢
I fixed my issue with the below config in next.config.js
, inspired by the previously mentioned solutions. But just in case people need an example of how it looks after it was changed to outputFileTracingExcludes
;
experimental: {
outputFileTracingExcludes: {
'*': [
'./**/node_modules/@swc/core-linux-x64-gnu',
'./**/node_modules/@swc/core-linux-x64-musl',
'./**/node_modules/esbuild/linux',
'./**/node_modules/webpack',
'./**/node_modules/rollup',
'./**/node_modules/terser',
],
},
},
outputFileTracing: true,
I struggled with this for the longest time because I had set outputFileTracing
to false
a long time ago, which still meant that files I didn't want were included, but I could not control it with outputFileTracingExcludes
. Also worth mentioning that I'm using pnpm, so thats why the locations are prefixed with **/
as the packages are nested further down in the store.
Verify canary release
Provide environment information
This is from a Vercel build. I've added the output below:
All serverless functions suffer this. Here's an example:
What browser are you using? (if relevant)
No response
How are you deploying your application? (if relevant)
No response
Describe the Bug
Our NextJS on Vercel no longer are deployable as of NextJS
13.0.2
due to exceeding the max payload size of Serverless functions.Here's an example of the basic 404 page (which does not do any Prisma-related queries):
Why are two versions of SWC core being included? I think that's the main culprit.
Here is an open ticket in the SWC project that highlights how both
gnu
andmusl
libraries are included. Is there anything we can do to pick a single one for Vercel builds?I noticed in the
13.02
release notes that Prisma being added to externals list. Could this be a side effect as well?Expected Behavior
Serverless functions do not include both
gnu
andmusl
and do not exceed the threshold for deployment on Vercel.NextJS version
13.0.1
succeeds.Link to reproduction
https://github.com/erawk/next-app-bug-report-serverless-functions-payload-size
To Reproduce
musl
andgnu
includedNEXT-1356