Yarn over npm?

[samhstn]

We are currently using just npm to manage our dependencies, but I think there are better alternatives such as yarn.

I have seen that we're not using it in any of our projects. Why is this?

I can't see any reason to not be using it

[nelsonic]

@shouston3 great question! (thanks for opening this discussion thread!) 😍

Yes, Yarn is on our radar (and there are good use cases for it, see below): https://github.com/nelsonic/nelsonic.github.io/issues/234 🤔 We haven't seen the need for it, yet. have you felt the need for it on one of your projects? e.g: do you have a project where installing dependencies takes longer than 20 Seconds? is it affecting your productivity e.g: by slowing down the "CI" build for a PR which means you cannot deliver your work on schedule...? These would both be legitimate reasons to consider using an "alternative" to NPM, using Yarn just because it's new and has a nice website with 3 Key Advantages (#Marketing101) is not (usually) a good reason to adopt something. if you're feeling pain using NPM that's different... 😢

I acknowledge that Yarn is "faster" than NPM for downloading packages. It employs a far more effective caching mechanism. no question there. I don't tend to re-install any dependencies in my project(s) very often unless there is a reason to upgrade something and then I would need to install a new version in any case, whereby Yarn offers zero advantage over NPM.

Unless you are working on many projects that have similar dependencies e.g. you're building several React components in separate repositories, publishing them individually to NPM and then including them back into a "main" project ... that would be a good use-case for Yarn. 👍 Most people @dwyl are working on one project at a time. Occasionally we "overlap" from one project to the next but rarely because context switching is cognitively expensive: http://geekswithblogs.net/MarkPearl/archive/2012/07/17/the-human-multitasking-problem-ndash-the-cost-of-a-context.aspx

Facebook are Creating an Artificial "Need" for a New Package Manager by using Bloated Dependencies instead of being Efficient ...

I can see the "need" for Yarn in the context of installing many large dependencies. however ...

@dwyl We don't routinely use mega-dependencies like "Babel" which we agree takes ages to install!!

So... I (personally) don't feel the need to install another global package on my system ...

Facebook are "solving" a "problem" they created by building huge modules in the first place and fragmenting the node.js community instead of (patiently) contributing back to NPM.

Thankfully, None of the projects/products @dwyl suffer from the "takes ages to install" problem. Because we build lightweight things that have as few dependencies as possible for both sanity and security reasons ... http://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/ ... 😧

What "FB Engineering" could (should) have done is take their Vast Capital and approach the lovely (talented and super dedicated!) people at NPM Inc (@isaacs and @seldo are super approachable and community-minded!) with a proposal ... for example:

"Hey NPM, we know you are over-worked trying to keep the most popular package manager on the Internet running and haven't got that much time for improvements ... would you consider letting us pay you guys a million bucks if you let us partner with you on improving NPM using all our node.js resources...?"

Preliminary Conclusion

We don't need Yarn, yet. Please feel free to monitor how long it takes to install your dependencies for a given DWYL project and if it goes above 15 seconds (or how ever long it takes you to read the next user story in the backlog for the project), leave a new comment on this thread so we can re-consider Yarn! 👍

Thanks!! ❤️ ✅

tl;dr (warning: contains lesson)

Yarn is technically "better" than NPM for the reasons described on the slick website... but it neglects to mention that the reason NPM is over-worked is because Babel, Webpack, React etc. are bloated packages which cause huge strain on the NPM infrastructure which is "free" (and always has been) to use for the community!! has any "node.js deveveloper" you know ever paid NPM a cent to use their service...?meanwhile they have hundreds of thousands of dollars in bandwidth and storage costs... but instead of helping to alleviate the problem e.g. by contributing more resources to NPM Facebook decided to re-write it and crucially re-brand it... that's the key part! re-branding...

Facebook (_and handful of other "Big" companies_) always want to own the system rather than use (and help improve) the already pretty decent system that exists. this is their way of trying to "brand" something that they did not invent and were not a part of creating but have decided to "appropriate" now because:

note: I did not create that "meme", it's pervasive on the net. just read: https://www.theguardian.com/technology/2016/dec/16/facebook-copies-snapchat-feature-for-15th-time imagine you're the developer working (really) hard to release a new thing/feature and Facebook just swooped in and shamelessly copied it often taking the "credit" for inventing it, how would you feel...? everyone who has been in Web development knows that Fb have systematically "claimed" every good idea that any other "social network" company has created. That's just lame.

There must be a reason "Zuck" was portrayed the way he was in "The Social Network", it's not just "Hollywood Drama" to "sensationalise" the events ...
https://youtu.be/VKnNUYknsuQ and https://youtu.be/xdiFzcpmmJc

FB have a culture of "improving" on other people's ideas, gaining a competitive advantage (because they have more capital to spend on customer acquisition) and then "fighting it out in court" which is always more difficult for the "little guy"... It worked for Microsoft against Netscape: https://youtu.be/m_2m1qdqieE (if you have never watched it, add it to your "watch later" list cause it's a "master class" in legal gymnastics, Bill Gates is unquestionably a genius and having studied "pre law" played a big role in the success of his software company!) ... and "Zuck" is every bit as "savvy" as Bill Gates was when it comes to using the best legal team to achieve his personal ambitions ...

There must be a reason why "Zuck" has a team of PR people "managing" his "public image": https://www.bloomberg.com/news/articles/2017-01-18/this-team-runs-mark-zuckerberg-s-facebook-page taking and carefully releasing the soft-focus photo op "official" photos ... maybe it's because of: https://www.theguardian.com/technology/2017/jan/23/mark-zuckerberg-hawaii-land-lawsuits-kauai-estate

This might not be "relevant" to the discussion about Yarn specifically, but it is relevant to the wider discussion about using "Developer Tools/Libraries/Frameworks" made by Facebook: https://github.com/dwyl/learn-react/issues/23 beyond discussing the technical merit of the tool we have to consider the context of it ... I don't have a "technical objection" to using FB tools, in fact I like GraphQL: https://github.com/dwyl/learn-graphQL

Let's evaluate Yarn on it's own merits. and determine if we have a need to use it. 👍

[iteles]

Thanks for raising this @shouston3 👍 :tada: A very legitimate question and I'm glad you've allowed us to have a place to address it! If you do think there is a need to move to Yarn, please don't hesitate to try it out and let us know as @nelsonic says :blush:

[samhstn]

@nelsonic thank you so much for the insight on how yarn probably came about, and from my subsequent reading around the topic, you are right that initially yarn came about from high volumes of dependencies and slow build times, but also nondeterministic installs (which myself and others have felt the unnecessary pain of) and security issues.

Although it has been interesting reading up on the history of the npm and facebook teams and how this tool came to be, I would prefer to focus on the pros and cons of using this tool.

Need?

I don't know why there is such a focus on the "need" for it, it is simply an alternative tool to use over the npm cli to improve your workflow.

I think a better question to ask would be, do the benefits of using yarn outweigh the costs?

With yarn, there is also practically no learning curve, you can be fully up and running in a few seconds (with a couple of seconds install time (brew install yarn) and a couple more seconds learning the interface and your good to go)

Also, any project that implements yarn won't force anyone to have yarn installed globally or to use yarn at all, they can still npm install and everything will work fine 👍 .

(But you still might as well run brew install yarn && yarn instead, as it would install faster and not fill up your terminal with stuff you will never read, etc).

To answer your specific questions:

Have you felt the need for it on one of your projects?

Not the "need", I can manage without it, but over the last few months I have been using it when I can because it speeds up my workflow with (as of yet) no downsides.

Do you have a project where installing the dependencies takes longer than 20 seconds?

For Dwyl projects, large install times are rare, but my last project took more than a minute to install and some other dwyl projects have similar install times

(Also perhaps this is caused by the specs on my macbook air, but running npm install in most projects takes longer than 20 seconds to run).

Is it slowing down your productivity?

Yes

However

Yarn is really new and your caution is definitely warranted (and your not alone here and here).

Although it is actively maintained, there is quite a lot of open issues.

And perhaps it is overkill for small projects

Conclusion

Could we try it out in one or two future projects and then evaluate if the benefits outweigh the costs?

[nelsonic]

@shouston3, good question. Why should we focus on a "need"...? 🤔

Time is finite. given the choice is between adding a user-requested feature vs. a dev technical task with limited user-benefit, the user story gets priority every time.

when making a change to an existing project we always focus on the User need, because otherwise we would end up with a project that gets re-written several times (at high expense to the client) without any direct benefit to the end-user... i.e: "FOCH" ...

so, we identify a need first and work from there. Not: "oh, there's this shiny new thing that has a slick website and so many "advantages" over the "old" (boring) tool, so, we should use it everywhere!!" 😕

it sounds like you might have a pain point on a slower-to-install project... document the pain with specific metrics and https://en.wikipedia.org/wiki/Root_cause_analysis

I understand the desire (or curiosity) to try a new tool and also acknowledge that Facebook added Checksums for "security", but remember that you are now trusting Facebook to control your dependencies ... the same company that sells your eyeballs & personal data to the highest bidder ...

"security" is a compelling "reason" FB are giving for using Yarn. They know they have to give a "USP" to get devs to hand over all their download/usage stats for NPM packages ... this is data FB will use to recruit the people who they discover as being the biggest users of their "stack"... it's a brilliant recruitment strategy. 💡 and anyone who does not see that is naive about FB's true intentions.

Read: https://www.facebook.com/help/152637448140583 do you honestly believe this though? If Facebook are not selling your personal data, why would Harvard University Press publish this book: https://www.amazon.com/gp/product/0674504577 Summarised in: http://uk.businessinsider.com/facebook-google-information-nsa-iphone-android-data-personal-2016-2

Do you think that Yarn is not sending all the usage stats for exactly what you are downloading and what you are working on to FB HQ?

Yes, the science museum is using Babel... https://travis-ci.org/TheScienceMuseum/collectionsonline#L503 and it makes me sad. 😢 but I didn't want to "argue" ... some people feel that ES6 adds "value" to their work... If you are asking me: would I do that again? the answer is a resounding: No! I would use https://github.com/dwyl/goodparts all-day-long and reduce the number of Huge dependencies to an absolute minimum.

If you want to try and use Yarn for The Science Museum CI and see if you can reduce the Build time from 11 mins I would be delighted!! 🎉
(on success this would be 100% welcome and I'd be stoked to pay an invoice for this work!)

Cost-Benefit Analysis

Yes, you will get a increase in speed installing dependencies using Yarn.

Please Quantify the loss in productivity in hours per day/week/month/year. and specifically note down all the instances where you have been unable to get any (other) work done because NPM has failed you. 😞

I install things maybe once every couple of weeks when there is an update to a dependency in a project I'm working. npm install (_and do something else like reading a PR/issue while it's doing it's thing, so I'm never waiting.

NPM works flawlessly for me. Long Live NPM! ❤️

By contrast I can see a glaring "downside" of installing a Global Tool Made by an organisation I don't trust:

while on the surface the tool sounds "more secure" because it has checksums... adding a "proxy" (man-in-the-middle for all my dependencies) violates one of the most basic laws of Computer Security: https://en.wikipedia.org/wiki/Trust_no_one_(Internet_security)

Also calling something "deterministic" sounds really sophisticated, but it's BS. I have been using NPM for 6 years and can count the number of times the Dependencies failed to install on one hand. If your experience is different please share which project you are feeling the pain on. (thanks!) 👍

Nothing against other people using Yarn. Each person has a different level of comfort/trust with Facebook. Personally I don't trust them. and giving them Global access to my laptop without a very good reason (i.e a need) is a definite "no no" for me.

Thankfully, this isn't a "problem" I'm going to have for much longer... ⏳ I'm absolutely delighted with Elixir! https://github.com/dwyl/learn-elixir 😍 and so incredibly excited to be using Elm https://github.com/dwyl/learn-elm I highly doubt Facebook will be meddling in Elixir/Elm at least for the foreseeable future given their investment in React, Immutable.js, Flow, Babel, Redux, etc. 🙃

Personally, I will continuewill using NPM on my localhost even if it's "slower" ... until I completely transition to elixir in the near future.

But everyone else is welcome to try Yarn and send their node package download/usage data to Facebook and act surprised when the FB "Technical Recruiter" emails them... 👍

It boggles my mind how people take NPM for granted. 😢 Sure Yarn looks/sounds "better" on the surface... but most people really aren't thinking through the "true cost" of routing all their dependency requests via https://registry.yarnpkg.com i.e. telling Facebook even more about you.

And while anyone can counter-argue that Facebook "doesn't care" or they can just read your dependency list from your package.json on GitHub to "mine" who is using their Stack ... I urge everyone to try and think about this from the sceptical perspective that maybe they are collecting more data on all developers in the community so that they can recruit people to work on their distraction machine. see: http://www.timewellspent.io/ and actually this is a brilliant way of collecting mountains of metadata on all developers in the Node.js community ... I'm not going to endorse Yarn without a very good reason/ "need". 👍

Love you Sam! thanks again for opening this discussion! ❤️

[samhstn]

That is not really a perspective I had thought about too much and as it seems to be a popular one and a definite downside, it makes sense to not implement yarn.

My issues lie with previous projects and can't really see these arising with the new stack.

I am going to continue to use yarn for personal projects as I literally have nothing I care about on my computer (should I still watch out?) and will happily ignore fb recruiters should they come knocking.

[varjmes]

I don't see how yarn is a better choice over npm in most instances. It is broken in a lot of ways. Yarns power is fast install of large amounts of packages, are you going to be installed 100s of packages a day repeatedly? Also the community around npm is way more responsive. Take a look at both the projects issue trackers. npm has a dedicated team, yarn is 1-2 people and some volunteers.

I don't like the idea of trying something new just because. npm is widely used and thus the barrier to entry on an informational stand point, is lower.

[samhstn]

Good points @Charlotteis, I think I'll put this issue out of it's misery

[isaacs]

@Charlotteis

I don't see how npm is a better choice over yarn in most instances.

Is that a typo? It seems like this sentence is incongruent with the rest of what you're saying in your post.

In any event, thanks for the generous props and/or candid criticism, as appropriate ;)

[varjmes]

@isaacs typo indeed!

I don't see how yarn is a better choice over npm in most instances.

[Neats29]

@nelsonic I got here from the discussion on the things-of-interest channel. We might also be switching to Yarn at work, and I've been reluctant as I've never had any issues with modules taking too long to install. So I appreciate the points raised here.

You mentioned:

... and giving them Global access to my laptop...

Could you please point me to any links that may explain this in more detail?

Thank you.

[nelsonic]

@Neats29 if you don't have a rule for who trust .then the choice of where you install executables from is a "personal decision" (i.e. up to each individual/team...)

Have a look at the installation instructions for Yarn: https://yarnpkg.com/en/docs/install#mac-tab

notice the sudo?

Look at how they install it on Ubuntu/Debian: https://github.com/yarnpkg/yarn/blob/d2865a91bc58c89a5e9e4e52a3b210e41e655add/scripts/bootstrap-env-ubuntu.sh#L6

Why sudo...? Why does it need "root" / "admin" access to my machine...?

Yarn is probably perfectly "safe" for now... but they are violating a basic security principal: https://github.com/dwyl/learn-security#principal-of-least-priviledge

A library that forces me to install with sudo is an immediate "No". Even if they claim to be "more secure", which as far as I can tell from reading the entire source code of the package when it was first announced, all they are doing is re-writing urls from HTTP to HTTPS and doing basic checksums which NPM already has.

Again, each person should make up their own mind if they want to use Yarn.

tl;dr

Sure, package caching is a useful feature, but the 3 "USPs" of Yarn could/should have been PR'd into NPM but instead Yarn (Facebook) have merely re-packaged (excuse the pun) the NPM's hard work and re-branded it so they can "own" after doing none of the difficult work of actually running a package repository!

They should have contributed the improvements up-stream instead of fragmenting the ecosystem.

Personally I'm way more interested in using "PETE" for all our future projects: https://github.com/dwyl/technology-stack/#the-pete-stack Our use of node.js is tapering off fast ... elixir is ❤️

I was really excited about node.js in 2011 because the idea of having "One Language" everywhere was compelling for beginner friendliness. But as time has passed (and dozens of new but pointless "features" have been added) it's lost it's shine. JS is not the most "elegant", "performant" or "self-documenting" language. We use JS because we have to, not because it's "the best". People are trying to make it "better" ... but it's really a case of sprinkling glitter ... JS it's becoming more like Java with each release (and that's not a good thing!) there are eight ways of writing a function now. Eight! Why?! 😧

Anyway... we all know it's possible to write simpler code. The feedback from @dwyl people who are learning Elixir/Phoenix is really positive:

Instead of debating wether to use NPM or Yarn, stick with NPM for Node projects and learn "PETE" in your "spare time" so you can web Apps/APIs that handle 4x the number of connections/requests compared Node.js/Express for the equivalent hardware. Elixir/Phoenix/Elm enable us to Write considerably less code that is much easier to test, maintain & extend! 😍

Don't Settle. Build Better! 🚀

[Neats29]

@nelsonic thank you for the detailed response.