NextJS static pages with dynamic routes results in oversized server.js

vbudovski commented 3 years ago

NextJS compilation target has been set to serverless. Fab builds without issues aside from extremely large server.js in resulting bundle. This prevents a deploy to Cloudflare as there is a 1MB worker size limit.

[…/input-nextjs] Finding all static HTML pages…
[…/input-nextjs] ✔ Found 88 static html pages.
[…/input-nextjs] Reusing NextJS renderer cache frontend/.fab/.cache/generated-nextjs-renderers.049a5d6.js
[…/input-nextjs] Finding all static assets
[…/input-nextjs] ✔ Found 238 assets.
[…/input-nextjs] Finding all public files
[…/input-nextjs]   ✔ android-chrome-192x192.png
[…/input-nextjs]   ✔ android-chrome-512x512.png
[…/input-nextjs]   ✔ apple-touch-icon.png
[…/input-nextjs]   ✔ browserconfig.xml
[…/input-nextjs]   ✔ favicon-16x16.png
[…/input-nextjs]   ✔ favicon-32x32.png
[…/input-nextjs]   ✔ favicon.ico
[…/input-nextjs]   ✔ mstile-144x144.png
[…/input-nextjs]   ✔ mstile-150x150.png
[…/input-nextjs]   ✔ mstile-310x150.png
[…/input-nextjs]   ✔ mstile-310x310.png
[…/input-nextjs]   ✔ mstile-70x70.png
[…/input-nextjs]   ✔ safari-pinned-tab.svg
[…/input-nextjs]   ✔ site.webmanifest
       [Builder] Building @fab/plugin-render-html:
[@fab/plugin-r…] Compiled 88 html files.
[…n-render-html] No fallback injected.
       [Builder] Building @fab/plugin-rewire-assets:
[@fab/plugin-r…] Inlining 117 assets.
[…rewire-assets] ✔ Done.
[…rewire-assets] Generating server code to rewire 135 assets.
[…rewire-assets] ✔ Done.
       [Builder] Build plugins completed in 3.21 seconds.
     [Typecheck] No Typescript plugins detected. Skipping.
      [Compiler] Compiling your server.js:
      [Compiler] Done in 8.61 seconds.
... snip ...
/server.js (12.9 MB)
     [Generator] Done in 0.17 seconds.
     [Generator] Zipping it up into a FAB:
     [Generator] Created fab.zip (9.38 MB) in 1.07 seconds

abdelhamid-attaby commented 3 years ago

Same issue here with nextjs 10.0.8 for a fresh project.

geelen commented 3 years ago

Right so this is a case we're not handing well, but I'm surprised we're handing it that badly...

Just out of interest, can you run tree -sa .next/serverless (brew install tree if you're on a mac) and paste the result:

It should look like:

.next/serverless
├── [       4078]  init-server.js.js
├── [       4078]  on-error-server.js.js
├── [        288]  pages
│   ├── [       3117]  404.html
│   ├── [       3103]  500.html
│   ├── [    1018934]  _error.js
│   ├── [        128]  api
│   │   ├── [     633420]  hello.js
│   │   └── [     633580]  time.js
│   ├── [         96]  background
│   │   └── [    1021815]  [size].js
│   ├── [    1024079]  dynamic.js
│   └── [       3842]  index.html
└── [        251]  pages-manifest.json

vbudovski commented 3 years ago

I've removed most of the pages for speed/readability and it still generates a 9.13MB server.js. The most interesting thing here I think is the size of the serverless _error.js. It bloats to about 7MB when all of the other pages get included, largely to do with next/dynamic loading I think.

frontend/.next/serverless
├── [      21687]  init-server.js.js
├── [      21687]  on-error-server.js.js
├── [        352]  pages
│   ├── [       3978]  404.html
│   ├── [       4011]  500.html
│   ├── [    3240598]  _error.js
│   ├── [      11480]  index.html
│   ├── [      12568]  pricing.html
│   ├── [      21035]  privacy-policy.html
│   ├── [      13061]  signup.html
│   ├── [      14900]  support.html
│   └── [      27786]  terms-of-service.html
├── [        299]  pages-manifest.json
└── [         21]  version.json

geelen commented 3 years ago

correct me if I'm wrong, but I don't see any dynamic routes? Can you run a next export and use @fab/input-static instead on the out dir?

vbudovski commented 3 years ago

Including dynamic routes (9.24MB):

frontend/.next/serverless
├── [      21689]  init-server.js.js
├── [      21689]  on-error-server.js.js
├── [        480]  pages
│   ├── [       4270]  404.html
│   ├── [       4303]  500.html
│   ├── [    3320244]  _error.js
│   ├── [         96]  confirm-email
│   │   └── [       3637]  [token].html
│   ├── [         96]  error-pages
│   │   └── [       6801]  maintenance.html
│   ├── [      10214]  index.html
│   ├── [      13211]  pricing.html
│   ├── [      20292]  privacy-policy.html
│   ├── [         96]  reset-password
│   │   └── [         96]  [uid]
│   │       └── [       3666]  [token].html
│   ├── [      12488]  reset-password.html
│   ├── [      13215]  signup.html
│   ├── [      14157]  support.html
│   └── [      27036]  terms-of-service.html
├── [        559]  pages-manifest.json
└── [         21]  version.json

@fab/input-static produces a server.js of 386KB.

                /server.js (386 kB)
     [Generator] Done in 0.04 seconds.
     [Generator] Zipping it up into a FAB:
     [Generator] Created fab.zip (1.84 MB) in 0.18 seconds

geelen commented 3 years ago

Wow ok. Do me one more test?

Delete all the .js files in your .next/serverless/pages dir (it's only _error.js, right?) and run fab build --skip-cache. Do you end up with something nearer the 386kb range?

(you can leave the dynamic routes there, I don't think they're the cause of the bloat)

vbudovski commented 3 years ago

Still off by a factor of 10, sadly: /server.js (2.85 MB)

geelen commented 3 years ago

Ok! I will investigate, thanks. (currently looking at what bloats the JS case so will circle back to this)

geelen commented 3 years ago

Can you please test something for me?

yarn build:fab && ls -l .fab/build

Note existing size of server.js

yarn add https://pkg.csb.dev/fab-spec/fab/commit/b34a9815/@fab/actions https://pkg.csb.dev/fab-spec/fab/commit/b34a9815/@fab/core https://pkg.csb.dev/fab-spec/fab/commit/b34a9815/@fab/input-nextjs
yarn build:fab && ls -l .fab/build

Did a) build succeed and b) how big is server.js now?

If it did build, does fab serve run it OK? Any routes now throw exceptions?

yarn fab build --minify && ls -l .fab/build

What's the final minified size?

vbudovski commented 3 years ago

Original build:

drwxr-xr-x 15 vitaly staff      480 Mar 17 18:10 _assets
-rw-r--r--  1 vitaly staff 16928784 Mar 17 18:10 server.js

Updated packages dont build for me:

      [Compiler] Compiling your server.js:
Circular dependency: node_modules/rollup-plugin-node-builtins/src/es6/readable-stream/duplex.js -> node_modules/rollup-plugin-node-builtins/src/es6/readable-stream/readable.js -> node_modules/rollup-plugin-node-builtins/src/es6/readable-stream/duplex.js
Circular dependency: node_modules/rollup-plugin-node-builtins/src/es6/readable-stream/duplex.js -> node_modules/rollup-plugin-node-builtins/src/es6/readable-stream/writable.js -> node_modules/rollup-plugin-node-builtins/src/es6/readable-stream/duplex.js
Use of eval is strongly discouraged, as it poses security risks and may cause issues with minification
Use of eval is strongly discouraged, as it poses security risks and may cause issues with minification
No name was provided for external module 'console' in output.globals – guessing 'console$1'
No name was provided for external module 'assert' in output.globals – guessing 'assert'
No name was provided for external module 'domain' in output.globals – guessing 'domain$1'
Creating a browser bundle that depends on Node.js built-in modules ('console', 'assert' and 'domain'). You might need to include https://www.npmjs.com/package/rollup-plugin-node-builtins
Build failed!

Errors encountered during Rollup build:

- Could not find module 'console' during build of 'frontend/.fab/.cache/generated-nextjs-renderers.81481bc.js'
- Could not find module 'assert' during build of 'frontend/.fab/.cache/generated-nextjs-renderers.81481bc.js'
- Could not find module 'domain' during build of 'frontend/.fab/.cache/generated-nextjs-renderers.81481bc.js'
- Could not find module 'console' during build of 'console?commonjs-external'
- Could not find module 'assert' during build of 'assert?commonjs-external'
- Could not find module 'domain' during build of 'domain?commonjs-external'

vbudovski commented 3 years ago

Also, I couldn't run the dev server on the original. (webpack5)

        [Server] 💎 fab serve 💎
        [Server] Reading fab.zip…
        [Server] ✔ Done. Booting VM…
Failed to load env from .env.local TypeError: e.info is not a function
    at processEnv (evalmachine.<anonymous>:74604:25)
    at Module.rGIR (evalmachine.<anonymous>:90004:9)
    at __webpack_require__ (evalmachine.<anonymous>:46557:29)
    at Object.1 (evalmachine.<anonymous>:49627:26)
    at __webpack_require__ (evalmachine.<anonymous>:46557:29)
    at evalmachine.<anonymous>:46614:20
    at Object.<anonymous> (evalmachine.<anonymous>:46616:7)
    at Object.<anonymous> (evalmachine.<anonymous>:97992:34)
    at __webpack_require__ (evalmachine.<anonymous>:11646:34)
    at Object.<anonymous> (evalmachine.<anonymous>:46533:27)
    TypeError: commonjsGlobal.process.cwd is not a function
error Command failed with exit code 1.

geelen commented 3 years ago

Damn, ok. Could you share your code with me somehow? Join the discord and DM me? https://discord.gg/Qvj3pJY

abdelhamid-attaby commented 3 years ago

@geelen I tested the above commands using a fresh Next.js project:

Before your commits:

After your commits without minifying:

After your commit with minifying:

geelen commented 3 years ago

Interesting, what fresh Next project template did you use? I didn't see __dirname in my testing at all

jtescher commented 3 years ago

@geelen can reproduce with:

$ npx create-next-app
$ cd example-app
$ npx fab init
$ npm run build:fab
 ...
 [Generator] Created fab.zip (1.6 MB) in 0.21 seconds

fab-spec / fab

NextJS static pages with dynamic routes results in oversized server.js #362