heroku / heroku-buildpack-nodejs

Heroku's buildpack for Node.js applications.
https://devcenter.heroku.com/articles/buildpacks
MIT License
4 stars 1 forks source link

Delete the .yarn/cache folder before packing the slug/after install #915

Closed osdiab closed 2 years ago

osdiab commented 3 years ago

Is your enhancement request related to a problem? Please describe. My company's repo became unable to deploy after switching to yarn 2 / berry with zero-installs, because the slug size was too large.

Therefore I added .yarn/cache to our .slugignore file so that it wouldn't be included in the slug. That made it work, but it also makes yarn install take a long time instead of 2 seconds like it does when it uses zero-installs (as well as losing the stability benefit of having checked-in dependencies instead of relying on registries to have everything intact).

Describe the solution you'd like Exclude or delete the .yarn/cache folder from the resulting slug. It's not necessary after the first install and it doubled my slug size, pushing us over the slug limit.

Describe alternatives you've considered Ignoring it in .slugignore, and enabling NODE_MODULES_CACHE=true. This is alright, but I'd prefer if we can leverage yarn 2's zero-installs instead of selectively disabling it in Heroku. But if this is the direction Heroku wants to go in, at least add that advice to the docs for migrating to Yarn 2, as well as debugging slug size.

osdiab commented 3 years ago

I found another issue: now that I did this, it works fine. BUT, when Yarn prunes devDependencies, yarn berry fails to find anything in the caches (probably since I slugignored it Nevermind, I un-slugignored it and it still does the same thing). so my builds end up re-fetching everything, defeating the purpose of caching:

Logs ``` -----> Pruning devDependencies Running 'yarn workspaces focus --all --production' ➤ YN0000: ┌ Resolution step ➤ YN0002: │ @every.org/api@workspace:packages/api doesn't provide monocle-ts (p2dc22), requested by io-ts-types ➤ YN0002: │ @every.org/api@workspace:packages/api doesn't provide newtype-ts (pf84c7), requested by io-ts-types ➤ YN0060: │ @every.org/api@workspace:packages/api provides pino (pf36a3) with version 6.6.0, which doesn't satisfy what @every.org/common requests ➤ YN0002: │ @every.org/common@workspace:packages/common doesn't provide monocle-ts (pcc3b6), requested by io-ts-types ➤ YN0002: │ @every.org/common@workspace:packages/common doesn't provide newtype-ts (pce918), requested by io-ts-types ➤ YN0002: │ @every.org/common@workspace:packages/common [1242a] doesn't provide monocle-ts (p3a2cc), requested by io-ts-types ➤ YN0002: │ @every.org/common@workspace:packages/common [1242a] doesn't provide newtype-ts (pc4f23), requested by io-ts-types ➤ YN0002: │ @every.org/website@workspace:packages/website doesn't provide @types/react (pbfc7e), requested by @n8tb1t/use-scroll-position ➤ YN0002: │ @every.org/website@workspace:packages/website doesn't provide monocle-ts (p7663e), requested by io-ts-types ➤ YN0002: │ @every.org/website@workspace:packages/website doesn't provide newtype-ts (p186a6), requested by io-ts-types ➤ YN0060: │ @every.org/website@workspace:packages/website provides pino (p91f41) with version 6.6.0, which doesn't satisfy what @every.org/common requests ➤ YN0002: │ @sentry/nextjs@npm:6.3.5 doesn't provide react (p13320), requested by @sentry/react ➤ YN0060: │ @svgr/webpack@npm:5.5.0 provides @babel/core (p7739f) with version 7.12.10, which doesn't satisfy what @babel/preset-env and some of its descendants request ➤ YN0002: │ @typeform/embed@npm:0.25.4 doesn't provide react-is (pfda01), requested by styled-components ➤ YN0002: │ isomorphic-webcrypto@npm:2.3.8 doesn't provide react-native (p13ee4), requested by @unimodules/react-native-adapter ➤ YN0002: │ isomorphic-webcrypto@npm:2.3.8 doesn't provide react-native (pa8d5a), requested by react-native-securerandom ➤ YN0002: │ isomorphic-webcrypto@npm:2.3.8 doesn't provide react-native-web (p4a961), requested by @unimodules/react-native-adapter ➤ YN0002: │ pg-query-stream@npm:4.1.0 doesn't provide pg (p97f7e), requested by pg-cursor ➤ YN0002: │ react-dev-utils@npm:11.0.4 doesn't provide typescript (p79ddf), requested by fork-ts-checker-webpack-plugin ➤ YN0002: │ react-dev-utils@npm:11.0.4 doesn't provide webpack (p2af19), requested by fork-ts-checker-webpack-plugin ➤ YN0000: │ Some peer dependencies are incorrectly met; run yarn explain peer-requirements for details, where is the six-letter p-prefixed code ➤ YN0000: └ Completed in 0s 847ms ➤ YN0000: ┌ Fetch step ➤ YN0013: │ @babel/code-frame@npm:7.0.0 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/code-frame@npm:7.10.4 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/code-frame@npm:7.12.11 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/code-frame@npm:7.12.13 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/code-frame@npm:7.5.5 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/code-frame@npm:7.10.4 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/code-frame@npm:7.12.11 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/code-frame@npm:7.12.13 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/code-frame@npm:7.5.5 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/code-frame@npm:7.8.3 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/code-frame@npm:7.12.11 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/code-frame@npm:7.12.13 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/code-frame@npm:7.5.5 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/code-frame@npm:7.8.3 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/compat-data@npm:7.12.7 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/code-frame@npm:7.12.13 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/code-frame@npm:7.5.5 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/code-frame@npm:7.8.3 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/compat-data@npm:7.12.7 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/compat-data@npm:7.13.15 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/code-frame@npm:7.5.5 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/code-frame@npm:7.8.3 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/compat-data@npm:7.12.7 can't be found in the cache and will be fetched from the remote registry ➤ YN0013: │ @babel/compat-data@npm:7.13.15 can't be found in the cache and will be fetched from the remote registry ```` and so on
joshwlewis commented 2 years ago

@osdiab there's a new buildpack version out today that I think should resolve your issues. You shouldn't need NODE_MODULES_CACHE=true, and maybe the .slugignore.

osdiab commented 2 years ago

Thanks for letting me know - I'm not working on this project anymore but I'll let my old coworkers know!

Omar

On Tue, Feb 15, 2022 at 6:34 AM, Josh W Lewis @.***> wrote:

@osdiab https://github.com/osdiab there's a new buildpack version out today that I think should resolve your issues. You shouldn't need NODE_MODULES_CACHE=true, and maybe the .slugignore.

— Reply to this email directly, view it on GitHub https://github.com/heroku/heroku-buildpack-nodejs/issues/915#issuecomment-1039587675, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAONU3SPH5KPPINMSFL6LVDU3FYODANCNFSM444AEQRQ . You are receiving this because you were mentioned.Message ID: @.***>

pistachiomatt commented 2 years ago

Sorry to revive this but @joshwlewis what was changed with that fix? I still see .yarn/cache in the slug consuming space.