Including getPayloadHMR in a NextJS app increases the bundle size significantly

domenico-ruggiano commented 4 months ago

Link to reproduction

No response

Describe the Bug

When using getPayloadHMR in a NextJS app, the bundle size increases significantly.

To Reproduce

Initial Setup

Install NextJS:

$ pnpx create-next-app@14.3.0-canary.70 --typescript .

✔ Would you like to use ESLint? … No / Yes
✔ Would you like to use Tailwind CSS? … No / Yes
✔ Would you like to use `src/` directory? … No / Yes
✔ Would you like to use App Router? (recommended) … No / Yes
✔ Would you like to customize the default import alias (@/*)? … No / Yes

Move all file into src/app/(app), tree should look like this:

$ tree --gitignore          
.
├── README.md
├── next.config.mjs
├── package.json
├── pnpm-lock.yaml
├── postcss.config.mjs
├── public
│   ├── next.svg
│   └── vercel.svg
├── src
│   └── app
│       └── (app)
│           ├── favicon.ico
│           ├── globals.css
│           ├── layout.tsx
│           └── page.tsx
├── tailwind.config.ts
└── tsconfig.json

Install Payload 3.0.0 with postgress adapter:

$ pnpx create-payload-add@beta

┌   create-payload-app 
│
◇   ────────────────────────────────────────────╮
│                                               │
│  Welcome to Payload. Let's create a project!  │
│                                               │
├───────────────────────────────────────────────╯
│
◇   ▲ Next.js  project detected!
│
◇  Install Payload in this project?
│  Yes
│
◇  Select a database
│  Postgres
│
◇  Enter PostgreSQL connection string
│  postgres://postgres:testpass@127.0.0.1:5432/testdb
│
◇  Successfully installed Payload and dependencies
│
◇  Payload project successfully initialized!
│
◇   Documentation  ────────────────────────────────────────────────────────────────╮
│                                                                                  │
│  - Getting Started: https://payloadcms.com/docs/getting-started/what-is-payload  │
│  - Configuration: https://payloadcms.com/docs/configuration/overview             │
│                                                                                  │
│                                                                                  │
├──────────────────────────────────────────────────────────────────────────────────╯
│
└   Have feedback?  Visit us on GitHub: https://github.com/payloadcms/payload.

Create a docker-compose.yml file to run a postgress db locally:

version: "3.9"

services:
  db:
    image: postgres:13-alpine
    environment:
      POSTGRES_PASSWORD: testpass
      POSTGRES_USER: postgres
      POSTGRES_DB: testdb
    volumes:
      - ./db_data:/var/lib/postgresql
    ports:
      - "5432:5432"

Then run docker-compose up -d.

Build Size

Running pnpm build, page / has approximately 92.7 kB of JS on first load:

$ pnpm build

Route (app)                              Size     First Load JS
┌ ○ /                                    5.25 kB        92.7 kB
├ ○ /_not-found                          904 B          88.4 kB
├ ƒ /admin/[[...segments]]               152 B           441 kB
├ ƒ /api/[...slug]                       0 B                0 B
├ ƒ /api/graphql                         0 B                0 B
├ ƒ /api/graphql-playground              0 B                0 B
└ ○ /my-route                            0 B                0 B
+ First Load JS shared by all            87.5 kB
  ├ chunks/009de13e-0259c6395e1b79aa.js  51 kB
  ├ chunks/5091-1d9cb54da3ef9f05.js      33.5 kB
  └ other shared chunks (total)          2.98 kB

Add a src/util/payload.ts file to connect to the Payload API:

"use server";

import { getPayloadHMR } from "@payloadcms/next/utilities";
import config from "@payload-config";

export async function getPayload() {
  return getPayloadHMR({ config });
}

And include it in the src/app/(app)/page.tsx file:

import { getPayload } from "@/util/payload";

export default async function Home() {
  const payload = await getPayload();
  return <h1>Hello World</h1>;
}

Now run pnpm build again, and the page / has approximately 353 kB of JS on first load, 4x the previous size:

Route (app)                              Size     First Load JS
┌ ○ /                                    464 B           353 kB
├ ○ /_not-found                          904 B          88.4 kB
├ ƒ /admin/[[...segments]]               155 B           442 kB
├ ƒ /api/[...slug]                       0 B                0 B
├ ƒ /api/graphql                         0 B                0 B
├ ƒ /api/graphql-playground              0 B                0 B
└ ○ /my-route                            0 B                0 B
+ First Load JS shared by all            87.5 kB
  ├ chunks/009de13e-0259c6395e1b79aa.js  51 kB
  ├ chunks/5091-1d9cb54da3ef9f05.js      33.5 kB
  └ other shared chunks (total)          2.98 kB

Bundle Analyzer

Install NextJS Bundle Analyzer running pnpm i @next/bundle-analyzer and next.config.mjs as following:

import { withPayload } from "@payloadcms/next/withPayload";
import bundleAnalyzer from "@next/bundle-analyzer";

const withBundleAnalyzer = bundleAnalyzer({
  enabled: process.env.ANALYZE === "true",
});

/** @type {import('next').NextConfig} */
const nextConfig = {};

export default withBundleAnalyzer(withPayload(nextConfig));

then run ANALYZE=true pnpm build.

It seems that using getPayloadHMR in the src/util/payload.ts file is causing the whole Payload UI to be included in the client bundle:

Screenshot 2024-05-21 alle 08 51 41

Payload Version

3.0.0-beta.34

Adapters and Plugins

No response

richardbutler commented 4 months ago

I'm seeing the same issue, but simply importing the getPayload function into a server component, which causes that entire lime green block to get included in the client bundle.

Repro repo is here: https://github.com/richardbutler/payloadcms-bundling-repro

(app)/page.tsx Root page with no payload imports:
- Bundles fine, no issues
(app)/[...slug]/page.tsx Article page with slug, which uses getPayload to payload.find a list of pages in the (app)/[...slug]/layout.tsx and also uses payload.find to get the page itself in (app)/[...slug]/page.tsx.
- Bundles the page and much of the payload admin bundle, including the (payload)/custom.scss, which causes the background colour to change on this page only.
- I've dropped an import 'server-only' into the service making the call, and it doesn't throw the exception that it would if it was being imported by a client component.

Route (app)                              Size     First Load JS
┌ ○ /                                    142 B          87.7 kB
├ ○ /_not-found                          903 B          88.5 kB
├ ƒ /[...slug]                           418 B           357 kB
├ ƒ /admin/[[...segments]]               151 B           446 kB
└ ƒ /api/[...slug]                       0 B                0 B
+ First Load JS shared by all            87.6 kB
  ├ chunks/3182-1b8285e46916339b.js      33.6 kB
  ├ chunks/4bd1b696-1585789b00318d32.js  51 kB
  └ other shared chunks (total)          2.95 kB

○  (Static)   prerendered as static content
ƒ  (Dynamic)  server-rendered on demand

✨  Done in 28.17s.

This only happens on production builds, not on development builds, both standard Next builds and standalone Next builds.

Reference: pages side by side

Reference: bundle analyzer (app)/[...slug]/page.tsx

Reference: bundle analyzer (app)/page.tsx

benyamynbrkyc commented 4 months ago

Had the same issue as well, managed to get it answered in the Payload Discord. Thanks to @AlessioGr and @r1tsuu for helping me resolve this issue. I'm just going to leave their comments from the Discord chat here, and claim no credit.

Alessio:

"Hey hey! This is because next automatically bundles all referenced client-side components to the page's JS bundle even if they are not actually used. Thus, even if you only use getPayloadHMR on a page to fetch data, the entirety of payload gets included in the JS.

This will need to be addressed in next, as there is no way to configure that behavior right now. We have an open PR in the next.js repo here: https://github.com/vercel/next.js/pull/65415

Once that's merged this will no longer be an issue! Would be great if you can give that PR an upvote 🙂"

Ritsu even created a gist for quickly patching Next.js in order to fix the bundle sizes. I managed to get it fixed thanks to it.

Gist: https://gist.github.com/r1tsuu/27cb56223d166620966edbcff44e6842

r1tsuu commented 4 months ago

Had the same issue as well, managed to get it answered in the Payload Discord. Thanks to @AlessioGr and @r1tsuu for helping me resolve this issue. I'm just going to leave their comments from the Discord chat here, and claim no credit.

Alessio:

"Hey hey! This is because next automatically bundles all referenced client-side components to the page's JS bundle even if they are not actually used. Thus, even if you only use getPayloadHMR on a page to fetch data, the entirety of payload gets included in the JS.

This will need to be addressed in next, as there is no way to configure that behavior right now. We have an open PR in the next.js repo here: vercel/next.js#65415

Once that's merged this will no longer be an issue! Would be great if you can give that PR an upvote 🙂"

Ritsu even created a gist for quickly patching Next.js in order to fix the bundle sizes. I managed to get it fixed thanks to it.

Gist: https://gist.github.com/r1tsuu/27cb56223d166620966edbcff44e6842

yes patch works like a charm!

domenico-ruggiano commented 4 months ago

Had the same issue as well, managed to get it answered in the Payload Discord. Thanks to @AlessioGr and @r1tsuu for helping me resolve this issue. I'm just going to leave their comments from the Discord chat here, and claim no credit. Alessio: "Hey hey! This is because next automatically bundles all referenced client-side components to the page's JS bundle even if they are not actually used. Thus, even if you only use getPayloadHMR on a page to fetch data, the entirety of payload gets included in the JS. This will need to be addressed in next, as there is no way to configure that behavior right now. We have an open PR in the next.js repo here: vercel/next.js#65415 Once that's merged this will no longer be an issue! Would be great if you can give that PR an upvote 🙂" Ritsu even created a gist for quickly patching Next.js in order to fix the bundle sizes. I managed to get it fixed thanks to it. Gist: https://gist.github.com/r1tsuu/27cb56223d166620966edbcff44e6842

yes patch works like a charm!

Thank you, it works perfectly! Just a note: you need to use Node v22 and if you are using src directory, you need to change next.config.mjs into:

const nextConfig = {
  experimental: {
    serverOnlyDependencies: [
      resolve(import.meta.dirname, "src", "payload.config.proxy.ts"),
    ],
  },
};

@r1tsuu May I ask how to create a patch for the new canary versions of Next.js, such as 14.3.0-canary.75?

Thanks for your support!

r1tsuu commented 4 months ago

Had the same issue as well, managed to get it answered in the Payload Discord. Thanks to @AlessioGr and @r1tsuu for helping me resolve this issue. I'm just going to leave their comments from the Discord chat here, and claim no credit. Alessio: "Hey hey! This is because next automatically bundles all referenced client-side components to the page's JS bundle even if they are not actually used. Thus, even if you only use getPayloadHMR on a page to fetch data, the entirety of payload gets included in the JS. This will need to be addressed in next, as there is no way to configure that behavior right now. We have an open PR in the next.js repo here: vercel/next.js#65415 Once that's merged this will no longer be an issue! Would be great if you can give that PR an upvote 🙂" Ritsu even created a gist for quickly patching Next.js in order to fix the bundle sizes. I managed to get it fixed thanks to it. Gist: https://gist.github.com/r1tsuu/27cb56223d166620966edbcff44e6842

yes patch works like a charm!

Thank you, it works perfectly! Just a note: you need to use Node v22 and if you are using src directory, you need to change next.config.mjs into:
const nextConfig = {
  experimental: {
    serverOnlyDependencies: [
      resolve(import.meta.dirname, "src", "payload.config.proxy.ts"),
    ],
  },
};
@r1tsuu May I ask how to create a patch for the new canary versions of Next.js, such as 14.3.0-canary.75?

Thanks for your support!

you can change the patch key in the package json (and pray that nothing was changed with this version in the patch diff context)

    "patchedDependencies": {
      "next@14.3.0-canary.75": "patches/next@14.3.0-canary.68.patch"
    }

richardbutler commented 4 months ago

@r1tsuu Legend. This works a charm, I owe you beer.

@domenico-ruggiano I manually applied the same changes on Next 14.2.3 (not canary) using yarn patch (instructions here). I'm also using Node 20 and everything is fine. I've added my patch file to the gist.

denolfe commented 4 months ago

Love to see everyone getting crafty in this interim period 👍

domenico-ruggiano commented 4 months ago

@r1tsuu Legend. This works a charm, I owe you beer.

@domenico-ruggiano I manually applied the same changes on Next 14.2.3 (not canary) using yarn patch (instructions here). I'm also using Node 20 and everything is fine. I've added my patch file to the gist.

@richardbutler my bad, I was using node v20.9.0. You're right it works perfectly also using Node v20 lts/iron.

Thanks for the info, I know how to apply a patch with yarn or npm. I'm just curious, how do you generated the git diff file? I tried cloning the repo with the patch and generating the diff file by comparing the pull request with the released version in the official Next.js repository (or visiting https://github.com/vercel/next.js/compare/canary...jmikrut:next.js:feat/%2350285-server-only-deps.diff), but I couldn't manage it because the differences were calculated on the source code and not on the "/dist/build/" folder.

domenico-ruggiano commented 3 months ago

@denolfe after upgrading payload to version v3.0.0-beta.49 (I'm currently on v3.0.0-beta.52) the patch doesn't work anymore.

After build, I'm getting this error:

Error: Could not find the module "./node_modules/.pnpm/@payloadcms+richtext-lexical@3.0.0-beta.52_@faceless-ui+modal@3.0.0-beta.0_react-dom@19.0.0-r_bcseew2il5vot4h3q2q2cxajke/node_modules/@payloadcms/richtext-lexical/dist/exports/client/index.js#IndentFeatureClientComponent" in the React Client Manifest. This is probably a bug in the React Server Components bundler.
    at ek (./node_modules/.pnpm/next@15.0.0-canary.37_patch_hash=liprbiohydtruwk26ivxoyce6e_react-dom@19.0.0-rc.0_react@19.0._vbcca4r3lbbuwgj3z4anxkkfoa/node_modules/next/dist/compiled/next-server/app-page.runtime.prod.js:84:14370)
....

denolfe commented 3 months ago

@domenico-ruggiano The patch is not officially supported. It will need to be redone by the author.

rilrom commented 3 months ago

I created a patch based on the PR the payload team have created in next, however it doesn't seem to work. Do we know if the PR is still working with the latest canary versions of next 15?

Edit: the patch works when I upgrade from 15.0.0-canary.53 to 15.0.0-canary.69. Unfortunately I now receive a similar error to @domenico-ruggiano when attempting to visit the admin panel. Once I remove the patch I am able to visit the admin panel again.

AlessioGr commented 2 months ago

This has been fixed by https://github.com/payloadcms/payload/pull/7620

github-actions[bot] commented 1 month ago

This issue has been automatically locked. Please open a new issue if this issue persists with any additional detail.

payloadcms / payload