Shrinkwrapping with jest takes 30 more seconds due to node_modules

tatemz commented 3 years ago

Expected Behaviour

Shrinkwrapping a node project should not take long and should not copy node_modules into build directory because they are ignored by .dockerignore and installed during the build

Current Behaviour

Shrinkwrapping can take up to 30 seconds on a project where most of the function handler's dependencies are devDependencies (e.g. jest)

Are you a GitHub Sponsor (Yes/No?)

Check at: https://github.com/sponsors/openfaas

[ ] Yes
[x] No

List All Possible Solutions and Workarounds

Add scripting to save and remove (stash) files/folders that you don't want copied before shrinkwrapping
Add scripting to re-add removed files after shrinkwrapping

Which Solution Do You Recommend?

I've proposed #888, but maybe there is a way to prefer using .dockerignore files when shrinkwrapping.

Steps to Reproduce (for bugs)

Make a new handler using a node template
npm i jest --save-dev to that handler
faas-cli build --shrinkwrap
Add more node_modules to see the copy process take longer

Context

Build time of 60+ seconds for simple projects

Your Environment

FaaS-CLI version ( Full output from: faas-cli version ):
Docker version ( Full output from: docker version ):
Are you using Docker Swarm (FaaS-swarm ) or Kubernetes (FaaS-netes)?
Operating System and version (e.g. Linux, Windows, MacOS):
Link to your project or a code example to reproduce issue:

tatemz commented 3 years ago

Hello! I'm just checking back in. I filed this issue as it was a real problem when developing locally. My CI pipelines have no issue since node_modules are in the .gitignore of my projects. I made MR #888 and have been using that build locally for all of my tooling. Would love to know what the next steps are to getting this issue shared with the broader community 😄

Thanks for a great tool! It's off to a great start!

alexellis commented 3 years ago

/msg: enduser

derek[bot] commented 3 years ago

Thank you for your interest in OpenFaaS. This project is maintained and made available for hobbyists and commercial users alike, so we need to balance our time across everyone's needs. Whilst we are excited in your interest in using OpenFaaS, we would also ask you to take a look at our contribution guide on Setting expectations, support and SLAs.

Commercial users can purchase support in order to get dedicated help from OpenFaaS Ltd, or they can book ad-hoc consulting hours to get an engineer to dedicate time to helping them.

If that is not a good fit for you at this time, please check out the OpenFaaS GitHub Sponsors options which are priced for practitioners like yourself. Organisations can also sponsor through their GitHub billing relationship.

When you become a sponsor as an indvidual, it will show this on your issues and PRs, so that the community can see that you are supporting our work, and can prioritise your needs.

If you are receiving this message after having received hands-on support from the OpenFaaS community, please join GitHub Sponsors to say thank you for our time.

Thank you for supporting OpenFaaS.

alexellis commented 3 years ago

Please see comment: https://github.com/openfaas/faas-cli/pull/888#issuecomment-934173327

alexellis commented 2 years ago

Just tried this on the Weekly Office Hours call and cannot reproduce the problem.

alex@alex-nuc8:/tmp/test$ faas-cli new --lang node17 lotsofstuff

alex@alex-nuc8:/tmp/test$ cd lotsofstuff/
alex@alex-nuc8:/tmp/test/lotsofstuff$ ls
handler.js  package.json
alex@alex-nuc8:/tmp/test/lotsofstuff$ npm i jest --save-dev

added 332 packages, and audited 333 packages in 11s

28 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

alex@alex-nuc8:/tmp/test$ time faas-cli build --shrinkwrap -f lotsofstuff.yml
[0] > Building lotsofstuff.
Clearing temporary build folder: ./build/lotsofstuff/
Preparing: ./lotsofstuff/ build/lotsofstuff/function
Building: lotsofstuff:latest with node17 template. Please wait..
lotsofstuff shrink-wrapped to ./build/lotsofstuff/
[0] < Building lotsofstuff done in 0.26s.
[0] Worker done.

Total build time: 0.26s

real    0m0.274s
user    0m0.065s
sys 0m0.221s
alex@alex-nuc8:/tmp/test$ ls
build  lotsofstuff  lotsofstuff.yml  template
alex@alex-nuc8:/tmp/test$

alexellis commented 2 years ago

@tatemz can you provide an example to reproduce this?

Thanks,

Alex

openfaas / faas-cli