search

harvard-lil / perma

Indelible links
408 stars 72 forks source link

Install js-wacz separately #3497

Closed rebeccacremona closed 2 months ago

rebeccacremona commented 2 months ago

This PR is a little peculiar.

It pulls js-wacz out of Perma's main package.json and creates a separate package.json in the services directory, for just that package.

It does this because, presently, nodejs is not installed on any production minions: we only need node packages to compile our frontend assets. (We commit the bundled js and css to the repo!)

To use js-wacz to convert WARCs to WACZs, we'll need to install nodejs on our worker servers, and we'll want to use the same version in prod as in dev. But... we don't necessarily want 222M of unneeded nodejs packages lying around on every worker, when all we are using is js-wacz. And, it might be confusing to see js-wacz listed amongst all our otherwise frontend-only packages since we are not using it on the frontend.

So, this splits them out, and installs js-wacz in a different node_modules folder as a separate step in the Dockerfile. The production installation can be informed by this approach, even though it won't be identical.

bensteinberg commented 2 months ago

The js-wacz node_modules on a test server takes up 26M.