Improve bundle size

[DavidRouyer]

Steps to reproduce

I'm building a project with Next.js, tRPC, Drizzle and Neon as a Postgres database (you can find the source code here: https://github.com/DavidRouyer/customer-service)

The library is used in the following way:

import { neon } from '@neondatabase/serverless';
import { drizzle } from 'drizzle-orm/neon-http';

import * as auth from './schema/auth';
import * as contact from './schema/contact';
import * as message from './schema/message';
import * as ticket from './schema/ticket';

export const schema = { ...auth, ...contact, ...message, ...ticket };

export { pgTable as tableCreator } from './schema/_table';

export * from 'drizzle-orm';

export const db = drizzle(neon(process.env.DATABASE_URL!), { schema });

I'm having an issue when deploying my application on Vercel, I'm getting the following message: The Edge Function "index" size is 1.04 MB and your plan size limit is 1 MB.

I've installed @next/bundle-analyzer to investigate the problem with the bundle size on the edge part:

As you can see, @neondatabase/serverless is bundled multiple times and represents a large part of my bundle:

Expected result

Have a smaller bundle size (or maybe a specific entry for edge only?)

Actual result

The bundle size is too big

Environment

Vercel, Edge Runtime

Logs, links

[jawj]

Hi @DavidRouyer. Thanks for filing this issue.

On your latest point, dist/serverless.mjs is not part of the npm package. The main bundled code in the npm package is in dist/npm/index.js, which is similar in size (at 140KB) to what you got by removing tests.

A key issue appears to be that at least 3 copies of our package are being included by the Next.js bundler. Do you think that's normal/unavoidable?

If you're only importing { neon } from the package, only a small fraction of the codebase should ever be used, so it's a shame that tree-shaking isn't eliminating the rest. I'll see if there's anything we can do about this.

[DavidRouyer]

Thank you for your answer!

On your latest point, dist/serverless.mjs is not part of the npm package. The main bundled code in the npm package is in dist/npm/index.js, which is similar in size (at 140KB) to what you got by removing tests.

Yeah I removed my comment because I was building the project with npm run build and not npm run export

A key issue appears to be that at least 3 copies of our package are being included by the Next.js bundler. Do you think that's normal/unavoidable?

If you're only importing { neon } from the package, only a small fraction of the codebase should ever be used, so it's a shame that tree-shaking isn't eliminating the rest. I'll see if there's anything we can do about this.

I don't know, I'm fairly new to to Next.js and everything is obfuscated, I'm just doing a standard next build. I'm looking for a solution to include the package only once, or tree shaking it.

[DavidRouyer]

I replaced the esbuild script with Vite (esbuild & rollup), and I got better results out of the box https://github.com/neondatabase/serverless/commit/a3acd190a1af3397240fb9271c72af75304a2323

✓ 114 modules transformed. dist/index.mjs 140.58 kB │ gzip: 38.93 kB (unminified) dist/index.umd.js 103.74 kB │ gzip: 32.86 kB

[jawj]

That's an initially impressive reduction ... but I think it's down to the fact that the vite command is missing anything equivalent to the --inject:shims/shims.js option we pass to esbuild. The output is thus likely to work in Node.js, but not in the serverless environments the package exists to support.

These shims are mainly pretty small. The problem is the buffer package, which also has to be bundled in.

Somewhat frustratingly, it seems that dead code elimination/tree-shaking is unable to remove quite a long list of unused methods on buffer — thing like readUIntLE. I've checked that terser and uglify-js also seem unable to identify these as dead code.

If you have any other ideas for addressing that, or otherwise shrinking the bundle size, do let me know. By skipping --keep-names and re-compressing with uglify-js after esbuild, I've so far managed to save about 10 kB, which may or may not be worth it.

[jawj]

I've looked at this a bit more, and not found any easy way to reduce the bundle size that doesn't have drawbacks.

If we remove methods from Buffer that aren't used, buffers returned by the library (e.g. for bytea columns) will be lacking expected methods.

And skipping --keep-names peppers error messages and stack traces with cryptic variable names.

[nicksrandall]

@jawj I was just coming in to report this issue -- glad it is already being discussed. For my specific use-case, I only need to use the neon export so it would be awesome if there was an entry to this package that only exports that function and nothing else (like Client and Pool).

Relying on your user's bundlers to do the magic of tree shaking isn't a great strategy IMO. It's better to provide the individual modules yourself if possible (where you can control the bundler).

I would love to be able to import { neon } from@neondatabase/serverless/neon` to get a much smaller bundle.

[jawj]

Thanks @nicksrandall. Unfortunately even the { neon } import pulls in a fair bit of node-postgres, in order to maintain compatibility by using its pre- and post-processing routines on the data that goes in and out of the database.

But I'll keep the issue open for a bit and see if there's anything we can do here.