Open paprikati opened 4 years ago
@paprikati Is it too much to exclude installation of all the devDependencies
? Or do you just need some of the devDependencies
in the build? If you set YARN_PRODUCTION
to true
, then the buildpack will only install dependencies
. (Docs: https://devcenter.heroku.com/articles/nodejs-support#only-installing-dependencies)
optionalDependencies
, as I understand them, are really for packages that may have an install error or may be inaccessible to the install environment (like a conflicting operating system), but the install does not exit with a failure. Docs here: https://classic.yarnpkg.com/en/docs/dependency-types#toc-optionaldependencies. I'm not sure if this is the use case it's meant for.
On the other hand, I could see this as a useful flag, but not one that should be implemented at the Heroku/platform level since it's a flag for the package manager. Have you thought about opening an issue with Yarn to support this feature?
So the problem I've got is that I need devDependencies
like webpack, but not stuff like storybook or jest.
Arguably a nicer solution would be a yarn buildDependencies
section or similar - so I'll open something with them, although not expecting a quick response.
It would be really useful to have some way in heroku of avoiding pulling storybook, jest etc. - would love any other suggestions
I have been thinking more about this, and I think it's tricky because the idea is that the devDependencies
vs. dependencies
should solve this - but now we have build dependencies and test dependencies, which fall under dev dependencies, but create bloat on Heroku builds. Therefore, this does make me think it may be a Heroku thing to figure out.
@paprikati can you provide me more details here? How much more time are you seeing on install times? If you add build dependencies (such as webpack) to dependencies
and skip install of devDependencies
, how much does your slug size increase? Thanks!
If I use devDependencies (rather than putting webpack etc. into dependencies
), then the overall build time is about 16s slower, which is about 10% of the total time (167s -> 152s) . It could be worse if I had a more complex testing infrastructure (e.g. an integration test harness like cypress).
Confusingly, if I put webpack etc. into dependencies
, and set YARN_PRODUCTION=true
then the slug size goes from 109Mb -> 95Mb, which I do not understand (it suggests that yarn prune is not working quite right, perhaps?).
To be totally honest, the way the JS ecosystem works right now, we would probably want to only be pushing a build artifact into the slug, rather than all the code (as you're essentially double storing everything), but I realise that's a bit scope-creep-ish from the original issue!
I agree with your final point 😄 especially when we get into compiled languages like TypeScript. There's a .slugignore
file that I think fits what you need. (It may be worth it for me to add something to Node.js Heroku docs too.) Here are the Heroku docs: https://devcenter.heroku.com/articles/slug-compiler
For the slug size, what is the NODE_ENV
set to? We use yarn install
to run an additional install for dependencies
because yarn doesn't have the "prune" functionality. (see here) If NODE_ENV
is not set to production
for the environment the above install is running, it will install all the dependencies (see docs) which could explain the larger slug size.
UPDATE: I wrote this and then realized that there's an earlier else that would catch NODE_ENV != "production"
. (https://github.com/heroku/heroku-buildpack-nodejs/blob/master/lib/dependencies.sh#L114) My guess is that it either has to do with the ignoring scripts in the pruning step or how yarn cache works. This is worth investigating though.
The "prune" step for yarn does absolutely nothing to dev deps from what I can tell
Lots of modern JS projects have devDependencies that you need in order to build the app (webpack etc.). But they also use big tools like storybook and jest which are not required at buildTime or runTime.
I would like to avoid installing these during my build process (because its slow). One way would be to put them as optionalDependencies in the package.json, and then run yarn install --ignore-optional.
Is this something the buildpack could support? I can see a few options: