search

sillsdev / web-languageforge

Language Forge: Online Collaborative Dictionary Building on the Web and Phone.
https://languageforge.org
MIT License
44 stars 29 forks source link

chore: Switch from NPM to PNPM #1811

Closed rmunn closed 6 months ago

rmunn commented 6 months ago

Is your feature request related to a problem? Please describe. I'd like to use PNPM rather than NPM for managing packages in Language Forge. I often want to have two copies of the LF source code on my hard disk, one pristine copy of the develop or master branch and another one with a branch checked out that I'm currently working on. With NPM, that means two copies of the nearly 300-megabyte node_modules folder. With PNPM, those copies would be linked and I'd have only one copy of each NPM package, saving me at least 300 megabytes. (And probably more, because any packages that are also used by LexBox or other projects would have only one copy on my hard disk). As the hard disk in my dev machine is a little tight for space, that extra 300 megabytes would help quite a bit.

Describe the solution you'd like Run pnpm import to convert package-lock.json to pnpm-lock.yaml, then find and fix any problems. (Which are rare, but at least once I've run into a package that assumed that node_modules had a specific layout, whereas pnpm uses a different layout in node_modules. 99% of the time it Just Works™.)

Describe alternatives you've considered Leaving things as-is with the extra space used up by those redundant node_modules directories.

Additional context PNPM used to be an obscure package manager, but it's very well-known by now. (See usage graph below. Source: https://npm-compare.com/pnpm/#timeRange=THREE_YEARS accessed 2024-05-10). The Svelte team uses it, as do many others, because reusing NPM packages usually means much faster runs of pnpm install. (Assuming your CI infrastructure has a PNPM cache set up, which isn't that hard to do if GitHub Actions doesn't have it by default).

image

rmunn commented 6 months ago

BTW, most problems I've seen when switching from NPM to PNPM were along the lines of "package foo not found" from package bar, which relied on foo but didn't have it in its package.json because package quux, always installed alongside bar, listed foo as a dependency. PNPM only gives packages access to the dependencies they explicitly list, which can avoid several possible security scenarios. The solution is at https://pnpm.io/faq#pnpm-does-not-work-with-your-project-here — you can add a PNPM hook to say "Hey, consider package bar to have a dependency on foo v1.2.3 even though bar doesn't list it" and then package bar will work.

I encountered one such problem when running pnpm import as an experiment last week. I didn't fix it at the time since I was busy with other things, but I believe that would be the fix, and it would be quite simple.

rmunn commented 6 months ago

Since PNPM uses links internally inside node_modules, if you copy node_modules from your dev machine into a Docker container, it can result in an error similar to the following:

1.749  ERR_PNPM_UNEXPECTED_STORE  Unexpected store location
1.749
1.749 The dependencies at "/data/node_modules" are currently linked from the store at "/home/rmunn/.local/share/pnpm/store/v3".
1.749
1.749 pnpm now wants to use the store at "/root/.local/share/pnpm/store/v3" to link dependencies.
1.749
1.749 If you want to use the new store location, reinstall your dependencies with "pnpm install".

We already want to move the npm install step inside the Dockerfiles anyway (and https://github.com/sillsdev/web-languageforge/issues/1653 tracks that desire), so I'll just solve #1653 as part of the PR that solves this issue.