search

sveltejs / kit

web development, streamlined
https://svelte.dev/docs/kit
MIT License
18.68k stars 1.93k forks source link

(Better) Node.js builtin emulation for client-side JS #8307

Open Leftium opened 1 year ago

Leftium commented 1 year ago

Describe the problem

I've found many NPM packages that almost work in SvelteKit client-side JS, but their access to node builtins like process and buffer results in errors. For example:

There is a work-around for these errors, but it's neither obvious nor simple:

Most average Svelte devs may just assume it's not possible (like I did at first).

Describe the proposed solution

Ideally SvelteKit would provide something similar to Parcel's Polyfilling & Excluding Builtin Node Modules

The SvelteKit DX would be greatly enhanced by integrating the work-arounds linked above as either default or optional configurations.

If the work-arounds are not included (by default), detecting this class of error and giving meaningful error messages pointing to the correct work-arounds would still be very helpful.

Alternatives considered

Importance

would make my life easier

Additional Information

Simple repro demonstrating problem:

bluwy commented 1 year ago

I think the core problem stems from Vite's stance of not polyfilling nodejs modules or syntax by default. The downside of doing this automatically is that it bloats the browser bundle. I don't think SvelteKit should change this, instead it uncovering the problematic dependencies is a good way to push the ecosystem towards not using nodejs modules.

Conduitry commented 1 year ago

Is there a straightforward way in Vite of opting in to pollyfilling? Is there something we want to link to from the Kit site? A commonly used Vite plugin?

madeleineostoja commented 1 year ago

I would also vote for just a more helpful generic error message along the lines of Sveltekit doesn't support Node.js APIs in the browser, use [X workaround](link) instead of bundling polyfills automatically, which I think is an antipattern for clientside code.

benmccann commented 1 year ago

For reference, the actual offending library in the reproduction is @apidevtools/json-schema-ref-parser. which is used by json-schema-to-zod and it looks like it's probably already been fixed: https://github.com/APIDevTools/json-schema-ref-parser/pull/280. There's an issue that json-schema-to-zod is depending on an older deprecated version of this package, which fetches a pinned version of the package forcing you onto the old broken one. I sent a fix to json-schema-to-zod here: https://github.com/StefanTerdell/json-schema-to-zod/pull/33

Leftium commented 1 year ago

Just an update for some context:

I originally ran into this issue on projects that were still using older versions of SvelteKit, where it seemed to be much more of a problem.

The current SvelteKit API makes it much easier to to import these modules from server-only code so the errors don't occur. 99% of the time, I think these troublesome modules can be called from server-only code like +page.server.js to populate PageData (which is then passed to the client-side code.)

So perhaps that is what a more helpful error message could suggest. (Along with the polyfill work-arounds if the modules are truly needed on the client-side.)

bluwy commented 1 year ago

Is there a straightforward way in Vite of opting in to pollyfilling? Is there something we want to link to from the Kit site? A commonly used Vite plugin?

There isn't otherwise everyone would opt-in to polyfiling and call it a day. Vite has some discussions that might help, but there doesn't seem to be one solid answer:

There is https://github.com/sodatea/vite-plugin-node-stdlib-browser which could work. Maybe a new FAQ entry would be nice.

dominikg commented 1 year ago

-1 to document polyfilling node only api to the browser as a valid option to work around badly built third party modules.

Documenting how to properly use serverside only code (which is done in a pretty consistent way already imho) and directing questions there should suffice. If at all we should add a warning that you should not polyfill node api to the browser except for very special usecases and if you have to wonder if yours is one of the special ones, it very likely isn't.

Rich-Harris commented 1 year ago

Yeah, documentation is the answer here — maybe an entry on the FAQ or a new 'Troubleshooting' question so that people can search phrases like 'process is not defined' or '__dirname is not defined'

bluwy commented 1 year ago

https://github.com/vitejs/vite/pull/11877 improves the error message a bit for Module "path" has been externalized for browser compatibility..., linking to Vite's troubleshooting docs. process is not defined is harder to catch as it happens anywhere in runtime. But with Vite's PR, I think it's enough and could consider this issue done.

Leftium commented 1 year ago

For reference, I just posted an answer on StackOverflow for a related question. I think this text would be a good candidate for the documentation:

Unfortunately, nodejs libraries (as opposed to browser or client-side libraries) like ccxt are not officially supported in client-side SvelteKit; they are only supported in server-side SvelteKit.

Generally, the best solution is to limit usage of nodejs libraries to server-side SvelteKit. This should be sufficient 99% of the time:

  • Call the nodejs library from +page.server.js and return values from the load function. The values can be accessed from +page.svelte via the data prop.
  • OR call the nodejs library from an API endpoint in +server.js. Then fetch() the appropriate internal API endpoint URL from +page.svelte (or almost anywhere else).

For the 1% of the time this is not sufficient, you will have to either polyfill nodejs built-in dependencies and/or remove nodejs dependencies from the library (ccxt) in question.

Sometimes polyfilling will get some nodejs libraries to work client-side. (Buffer is a nodejs built-in module):

  1. To fix local dev errors.
  2. To fix build-time errors.