Competing lockfiles create poor UX

[jmorrell]

NB: I'm creating this issue in the yarn repo, but this is actually a shared issue between yarn and npm.

With the release of npm 5 back in May, the Node ecosystem now has two lockfile-based package managers. This was overall a win for users, and it has been good to see competition in this space.

However now that there are two competing lockfile formats this can create a new problem for users, especially beginning developers.

When npm 5 came out, Heroku added a new failure if an application was submitted with both npm and yarn lockfiles. In the past few months this has become the most common reason Node builds fail on Heroku and these failures account for ~10-12% of all Node build failures on the platform. Thousands of developers hit this every month.

It's surprisingly easy to end up with both in your repository. Even as an experienced developer I've found myself running the wrong tool for a specific project and having to catch myself before commiting. For an inexperienced developer building their first server / react / angular project who might not understand what a package manager, lockfile, or git repo is, this can be highly confusing.

Neither tool will give a warning or error if the other lockfile exists:

❯ ls *lock*   
ls: *lock*: No such file or directory

❯ npm --version
5.8.0

❯ yarn --version
1.5.1

❯ npm install
npm notice created a lockfile as package-lock.json. You should commit this file.

added 659 packages from 437 contributors in 10.553s

❯ yarn install  
yarn install v1.5.1
info No lockfile found.
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
success Saved lockfile.
✨  Done in 7.67s.

❯ ls *lock*          
package-lock.json yarn.lock

This is likely especially true for Yarn users where most of the documentation on the web instructs users to npm install. Users who copy + paste commands from docs or Stack Overflow are likely to end up here.

I've spoken to @zkat and @iarna at npm and @arcanis on yarn, and all agreed that this is an issue that should be addressed, but there was not full agreement on how. Ideally this issue prompts discussion and both tools can agree on how we can help users here.

I've compiled a list of potential mitigations that have been suggested to me:

Potential solutions

Do nothing

Is there a technical reason a user might want two lockfiles? In this case, how can external tools determine which package manager should be used for that application?

Error if the other lockfile exists

Yarn could print an error and exit if package-lock.json exists and vice-versa.

Pros:

• simple and easy to implement
• the user gets an error where they are in a position to immediately fix it

Cons:

• Not a fantastic user experience

Convert the other lockfile

Yarn could read package-lock.json, convert it into yarn.lock, and remove package-lock.json. npm could do the opposite.

Pros:

• great user experience
• users switching tools would not get a new set of dependencies as a side-effect

Cons:

• My understanding is that due to the different dependency resolution strategies this conversion would be lossy both ways
• Requires each tool to add and maintain code to understand the other lockfile
• Lockfile formats may change over time

Delete the other's lockfile

Just remove the other lockfile and create a new one

Pros:

• effectively prevents this situation

Cons:

• surprising behavior
• user gets new set of dependencies

Run the other tool for the user

If yarn sees a package-lock.json but not a yarn.lock it could log a warning and call npm install for the user

Pros:

• User gets what they wanted

Cons:

• Surprising behavior
• somewhat complicated

Add config to package.json to indicate

Add a field in package.json to indicate which package manager a project should use

"package-manager": "yarn"

Pros:

• Explicitly conveys the user's intent

Cons:

• Adds more config for the user

Other?

Maybe I'm missing something that would work better

[imsnif]

Answered this in the other issue - I ask that the discussion move there so as to keep this issue on topic. Thanks!

[nottrobin]

I was carefully watching and participating in https://github.com/yarnpkg/yarn/issues/3614 (with currently 254 :+1:s). That issue has now been closed and the discussion moved here.

Ignoring practical challenges, in my opinion by far the best UX would be provided by an option not mentioned in the summary at the top, but mentioned by @thatlittlegit:

I think that the end goal should be to have one lockfile format shared among Yarn and npm.

Which is also backed up by @BrainBacon's point:

It seems odd to me that any kind of project should specify what package management tool should be used.

Again, ignoring practical challenges for a minute, the theoretical counter-argument to this was made by @iarna in this thread (and @jhabidas in the other thread):

There are a ton of advantages of having multiple actively used tools that make different tradeoffs because it means user use cases can be better met than can be with a single general solution.

Personally, I don't think this argument should apply in this case, as I made in a comment on the other thread (with 95 :+1:s, 2 :-1:s):

Yes I agree, Yarn wants to maintain its independent space as far as possible, so it has the flexibility to provide a better experience. Which is why, for example, Yarn's CLI is deliberately not compatible with NPM's.

However, I think lock files are outside of the space where Yarn can reasonably maintain independence. package-lock.json and composer.lock are committed to the repository, alongside package.json and composer.json. These are not tool-specific files, they are instead project-specific files specifying the exact dependency versions that the project is guaranteed to work with.

...

For Yarn to be a useful tool, it has to allow developers to follow standard models in their projects, so the project can remain tool-agnostic. Which, after all, is why Yarn was built on top of package.json and not its own separate dependency file.

Having read most of this thread, I still agree with my initial assessment - if Yarn continues to consume the same package.json dependency file from the repository, ideally it should emit the same package-lock.json file into the repository, so the repository can remain tool-agnostic. If Yarn read its dependencies from yarn.json instead of package.json I wouldn't keep making this point.

I believe that we should admit that this is the ideal situation, the seamless user experience. However, I do understand @iarna's other point:

A single lockfile format is not, I think, a likely outcome as there are different tradeoffs between the lockfile formats and lack of consensus as to which one's are best

(although I disagree that "having both tools being able to mutually understand one another's lockfiles is sufficient").

If the ideal scenario of allowing repositories to be tool-agnostic is definitely not achieveable (and I see the many comments about how the different lockfiles are built in fundamentally different ways and contain different information), then where we appear to be going seems just about palatable, which is:

• NPM/Yarn will emit warnings if they spot the other's lockfile (and maybe a config option to make this into an error)
• Both tools provide mechanisms for converting one lockfile into another

I don't think either should, under any circumstances, convert and delete the other's lockfile. This would mean that new developers would be forever converting from one to the other and back, depending on what tool that developer happens to use (given that too many people commit with git commit -a).

I also don't agree with @imsnif's point:

I don't think there's a good reason for a single package to have both a package-lock.json and a yarn.lock file.

I think it's presumptuous to dictate what a developer runs on their computer, or in their production environment. It's at least courteous not to be more restrictive than you need to be. If your project is only ever run by your team of developers, and they all have exactly the same software running, great. But you may have a dev who finds it much easier to use NPM than Yarn for some reason. And especially if you make open source software, you want to make it as easy for community members to get up and running as possible. Given that NPM and Yarn are simply tools for installing the same dependencies from package.json, they should just work. A dev should be able to see package.json, have a tool to interpret it, and not need to worry further. Which used to be the case before this lockfile conflict came alone.

[imsnif]

Hey @nottrobin - I think I understand where you're coming from. If we ignore practical challenges and philosophical differences, it would definitely be better for the various tools' users if they all consumed the same lockfile (this is my personal opinion, of course).

But since we can't ignore said practical challenges and mostly seemed to have agreed the philosophical differences have their place, I think the compromise we came to (the one you outlined above) is definitely "good enough".

Do note that a hidden assumption in said compromise is that the tool choice is one made per project rather than per developer's machine or production environment.

[nottrobin]

Do note that a hidden assumption in said compromise is that the tool choice is one made per project rather than per developer's machine or production environment.

Yes I think this is what most concerns me - although I think in the scenario I outlined above it is still possible to have a project support both Yarn and NPM.

I do think that ultimately that if the two projects abandon the mission of trying to allow a project to be tool-agnostic then really they should stop sharing a dependency file. Yarn should switch to reading yarn.json rather than package.json. Anything else is just messy and confusing.

[nickserv]

If npm lockfiles already have a superset of the dependency relationships that Yarn lockfiles support, why not support both in Yarn? Yarn could switch to package.json and any additional fields that may be desired by Yarn could be added in the future (similar to how some SVG editors like Inkscape manage editor metadata). Then Yarn could have the same dependency format as npm and be back compatible with yarn.lock, converting npm lockfiles to whatever structure Yarn wants in memory lossily.

[imsnif]

I do think that ultimately that if the two projects abandon the mission of trying to allow a project to be tool-agnostic then really they should stop sharing a dependency file. Yarn should switch to reading yarn.json rather than package.json. Anything else is just messy and confusing.

Perhaps, perhaps not. Certainly, this would require a massive breaking change of the tool (yarn). Less radical changes with more direct and measurable benefits have been passed over.

I'm not saying it's a bad idea, I'm just saying I don't feel it's a practical one.

[nottrobin]

I'm not saying it's a bad idea, I'm just saying I don't feel it's a practical one.

I agree that it seems like a big ask. But what I'm saying is that this seems like a pivotal issue for the project.

Up until now, Yarn has been a tool that developers could choose to use instead of NPM to install Node dependencies for any project with a package.json. Now, as you point out, it's requiring projects to explicitly choose between Yarn and NPM. That's a huge change, and shouldn't be made accidentally. This is the crunch point to make that call.

To my mind, there are 3 ways forward:

1. Continue to be a drop-in replacement for NPM by finding a way to align Yarn with NPM on all project-level interfaces (standardise the lockfile)
2. Deliberately diverge from NPM by using explicitly different project-level interfaces (e.g. yarn.json and yarn.lock)
3. Double down on providing half of NPMs interface, and a half different interface. This is actually the same as point 2., but while looking to most people like point 1.

I believe the third option here confuses the whole Node space, and weakens both tools considerably.

[nickserv]

It can still be back compatible. Yarn could just have a built in npm lockfile convertor so when it sees package-lock.json, it will keep it there and convert it to yarn.lock's format in memory. As far as I'm aware, npm can't do this because Yarn's lockfile has less information than npm's, so in my opinion it would be best for Yarn to standardize with npm.

[Vinnl]

@nottrobin I think you're mostly correct in your analysis, however:

Now, as you point out, it's requiring projects to explicitly choose between Yarn and NPM. That's a huge change, and shouldn't be made accidentally.

I think this has always been the case, or at least w.r.t. the primary benefit that Yarn was initially touted to bring: reproducibility of dependency trees. You can check in your yarn.lock, but that is practically useless if other developers add/remove dependencies without respecting that lockfile.

[imsnif]

@nottrobin - I agree with @Vinnl. As I mentioned above, while I don't want to be in a position of telling anyone how they should work, I think using both yarn and npm for installing dependencies in the same project is an antipattern.

Even though both tools could technically put in a lot of work to make this work, I don't feel this is something we as maintainers should encourage. There are countless other benefits to having interchangeable lockfiles (as the various discussions in the different threads show) but I don't think this is one of them, personally.

[nottrobin]

but that is practically useless if other developers add/remove dependencies without respecting that lockfile.

Yes, I suppose to some extent Yarn was in that muddy water from the start - the existence of yarn.lock did already mean it partially had its own interface. I think that was always a bit of a messy thing to do, which served Yarn's purpose of wanting people to go from NPM to Yarn, but not back.

But it was a decision I felt more comfortable making for my project, because at least I knew that NPM was still entirely supported - since it didn't provide locking in the first place, it would continue to work as well as it always did.

Now that's changed, because NPM does lock dependencies. If I choose to bind a project to Yarn, I will now be keeping yarn.lock up-to-date, and not package-lock.json, so it is no longer true that someone can use NPM effectively on my project.

[Spongman]

It sounds like you're saying that yarn is no longer intended to be a drop-in replacement for npm? That it's only supposed to be used by yarn-only projects?

I think if that's the case, then you owe it to everyone to make this fact clear on the yarn homepage - use this or npm, not both.

[arcanis]

It sounds like you're saying that yarn is no longer intended to be a drop-in replacement for npm?

It never was. The interface somewhat mimicked npm to make it easier for users to understand how to use it, but from the very beginning some things worked differently. The main reason some people thought it was a drop-in replacement was that npm was just lacking the features to compare.

Now that npm is catching up on some aspect and decides to implement things differently, it simply starts to show that we have different approaches and make different decisions (for example the recent offline mirror feature npm implemented isn't compatible with our existing one). So in short: it never was "safe", it just happened to accidentally work.

I think if that's the case, then you owe it to everyone to make this fact clear on the yarn homepage - use this or npm, not both.

As a matter of fact, we do have migration instructions. Your remark made me notice that it unfortunately contains a wrong paragraph that might give people the wrong impression 🙁 We would gladly accept a PR to change this paragraph to something more in line with our recommendations (ie. to use Yarn consistently across teams, in order to make sure that everyone can benefit from the various features that might be used by the project).

[Spongman]

It never was

err... you need to talk to your marketing people, then. https://yarnpkg.com/lang/en/docs/

Yarn interops directly with many features of npm, including its package metadata format, allowing for a painless migration.

[arcanis]

We don't have marketing people, but we do accept nice PRs 🙃

In this particular case it doesn't sound too wrong though. We do interop with many features, just not with all of them, and the migration is painless in most cases (at worse it's a yarn import away).

[Spongman]

migration is painless

i'm not really interested in migration. i'm looking for what was promised here (these guys definitely have marketing people): https://code.fb.com/web/yarn-a-new-package-manager-for-javascript/

It has the same feature set as existing workflows while operating faster, more securely, and more reliably.

yarn today is not that.

AFAICT there are 4 classes of users here:

• those that only use yarn in their projects,
• those that bought the facbook sales-pitch (above) but haven't yet hit any of these lock issues,
• those that are painfully trying to work around the incompatibilities by manuall converting lock files when necessary, or using other hacks to get it to work,
• those that just gave up on yarn and switched back to npm.

i really don't want to be in the last category, but it seems that's where i'm being pushed.

[zkat]

@Spongman what's stopping you from that last one? We can probably fix it ;)

[Vinnl]

@Spongman I'm not affliated with Yarn, so I guess I can be a little more blunt: there's really no point in stating in this issue that you think the wording is wrong. If you think the wording is wrong, go to that page in GitHub, click the Edit button, and submit a pull request with better wording. arcanis made it clear above that they're open to that.

(I guess you probably can't edit the blog post, but the website is probably the most important here.)

[nottrobin]

I can see from @imsnif and @arcanis' responses that the official position here seems to be that Yarn was "never intended" to continue to work seamlessly alongside NPM.

But I totally agree with @Spongman that this is the impression Yarn gave, and I really don't think that was an accident at the time. That was its genius - you could have improved speed, security etc. while still completely supporting the official NPM standards.

In either case, this issue makes Yarn's position on this much more clear for me than it was previously, and of course, Yarn's maintainers can choose to take it whatever direction they choose.

But I think you're drastically under-estimating the number of people who used Yarn precisely because they believed it maintained compatibility with NPM (at the project level), and would never have made the switch otherwise. I believe that the 254 :+1:s and 10 :heart:s on https://github.com/yarnpkg/yarn/issues/3614 and the 57 upvotes on "Should I commit yarn.lock and package-lock.json files?" make this abundantly clear.

If Yarn abdicates any responsibility on that front I think it will lose not only @Spongman and my teams, but many others besides.

[arcanis]

Frankly I really don't understand the problem you have with using only Yarn, or only npm. What you're saying is basically "hey, I can't force my team to use Yarn, so I'll force them to use npm". That doesn't make any sense to me. If you use Yarn's features then make everyone use Yarn, and if you want to use the npm features, then make everyone use npm. It's as simple as this.

Any inbetween means that at the very least your builds are not consistent accross your team, which goes against the premise of Yarn, as mentioned before. One of your engineers could start using workspaces, and it would work, but would break on npm. Same of the offline mirror. Same for the dependency resolutions. Even worse, since some fields are entirely unknown by npm, it would silently do the wrong thing.

As for the communication, the fb blog post doesn't mention a drop-in replacement. Let me quote the part where Yarn is introduced. It literally says that it replaces the workflow. I guess you got confused by the "remains compatible with the npm registry", which is a fair point you should bring to npm, not to us (there's the npm cli, the npm registry, and of course npm Inc itself).

Yarn is a new package manager that replaces the existing workflow for the npm client or other package managers while remaining compatible with the npm registry.

---

what's stopping you from that last one? We can probably fix it ;)

@zkat That's helpful, thank you.

[imsnif]

@nottrobin - I can't speak for yarn's original intentions because I wasn't around at the time. I was however working in a mixed yarn/npm environment with several dozen repositories.

I can say that it was perfectly clear to all developers involved back then that the choice of yarn/npm was a per-repo choice, just like the choice of express/hapi, mobx/redux, etc. This became even more clear as npm@5 came out with its own lockfile format and some developers decided to start using it with new repositories.

When a developer would install a dependency with the wrong tool, it would create a mess even before npm@5, because that dependency would not be consistently locked. It caused problems in our various environments and it was very clear to all involved that this was a mistake*.

I realize this might be confusing - and I understand this may not be 100% clear to everyone through no fault of their own, but I don't feel it's right to drastically change the tool to fit that misunderstanding. The way I see it, yarn is a drop in replacement for npm on a per-repo rather than per-box basis.

*Granted, that particular situation of multiple non-workspaced repos with mixed yarn/npm tooling was not ideal on a Human level rather than a tech level for exactly those mistake potentials and it was eventually fixed. I use this example here to show the two tools can work side-by-side if approached correctly.

[nottrobin]

Frankly I really don't understand the problem you have with using only Yarn, or only npm.

Yes, you @arcanis and @imsnif have made clear that you don't understand. The only point I'm making is that lots of people (look at the :+1:s) made the same "mis-"interpretation and want Yarn to work alongside NPM, regardless of whether you understand that or not. If Yarn is not the tool for us, so be it.

(Just one final point - @imsnif it's entirely ridiculous to compare a tool for installing Node dependencies to a project choice like express vs hapi, mobx vs redux. Those are fundamental characteristics of your app. If you can't see the obvious difference, no wonder you don't understand my point.)

Anyway, I'm done now. I think I've made my point as well as I can.

[Spongman]

For open source projects its never a question of forcing "the team" to use yarn or npm. It's a question of what is most convenient for everyone. In the real world npm is king so, unless yarn can play nice in an npm world, then it's dead.

[nottrobin]

^ This

[arcanis]

I'm afraid that's a misconception as well. During the past year, Yarn went from 20% of the total amount of requests to the npm registry to 40% in April. Last time I heard the stats (straight from the npm folks, since our stats recently broke when the npm registry went to Cloudflare), it was about 48% of the requests. In the actual real world, Yarn and npm coexist, plain and simple.

I'd now appreciate if we could go back to the topic at hand, which is: who's willing to write a PR to write a warning when we detect a package-lock.json file at install time? Thanks.

[imsnif]

Psst, @arcanis - https://github.com/yarnpkg/yarn/pull/5920

[arcanis]

Awesome :heart_eyes:

Between this and your work on yarn import, can we close this issue then?

[imsnif]

I thought we'd wait for @jmorrell to give us some stats of how this affects the problem on Heroku so that we can decide if this is enough or we'd like to change something (warning/error, etc.) wdyt?

EDIT: afaik the warning has not been released yet, so we still have some time to wait.

[Spongman]

I'm afraid that's a misconception as well.

How so? Volume of traffic at npm tells you nothing about whether or not the project is open source.

a more accurate assessment would be to count which github projects have 0,1 or 2 of (yarn.lock, package-lock.json). personally i have never seen an open-source project (of any appreciable size) on github that has a yarn.lock file and no package-lock.json (except the obvious).

IMO if you're going to keep separate lock files, then BOTH package managers should detect the other's lockfile and ERROR (possibly with an override flag).

[imsnif]

All - I'd kindly ask that we stay on topic and take any non-directly-related comments offline (eg. our Discord channel). There are a lot of people subscribed to this thread and in fairness, I feel the npm<>yarn discussion does not belong here. Thanks!

[rzr]

I havent read the whole tread, but IHMO user should be at least notified that there is an ambiguity:

if npm sees a yarn.lock file then npm should print something like: "WARNING: this project seems to use yarn, maybe you should use 'yarn' instead of 'npm install'"

if yarn sees package-lock.json file then yarn should print something like: "WARNING: this project seems to use npm, maybe you should use 'npm install' instead of 'yarn'"

And as suggested above, a way to "prefer" package manager (in package.json).

[hsribei]

I can say that it was perfectly clear to all developers involved back then that the choice of yarn/npm was a per-repo choice, just like the choice of express/hapi, mobx/redux, etc.

This was not clear to me at all. Up until now I thought it was a developer-local choice, and multiple developers could coexist on a single repo using whichever one they liked, while it being a little more convenient to use only one consistently. The express/hapi etc examples are good though and make it clear that that's not a choice. This should have more visibility.

Add a field in package.json to indicate which package manager a project should use

"package-manager": "yarn"

I think this is the best solution, iff it's implemented in all package managers.

The package managers then MUST 100% refuse to proceed if they're called on the wrong project, just as you would expect an error if you required redux but then tried calling mobx functions on it. They should issue an error, not a warning, and inform the user how to proceed with the correct package manager.

[iarna]

Add a field in package.json to indicate which package manager a project should use "package-manager": "yarn"

I think this is the best solution, iff it's implemented in all package managers.

If you want npm to consider this I encourage you to open a discussion in the appropriate venue, but I'll say that I don't like this direction. I would like to see convergence between package managers not divergence.

[jasonkarns]

I would recommend against some new "package-manager" field, instead recommending the existing engines stanza which already has prior art.

Both the yarn and npm docs mention (in passing) usage of engines.npm and engines.yarn to specify the versions of each that are expected to be used. Yarn will even error out if the yarn version doesn't satisfy engines.yarn.

https://yarnpkg.com/en/docs/package-json#engines- https://docs.npmjs.com/files/package.json#engines

Even if neither npm or yarn check engines for the "competing" manager (which would be nice), using this field is still a great way to communicate to other developers which manager is expected to be used. (if the existance of package-lock.json or yarn.lock isn't enough of a clue)

[Spongman]

I would like to see convergence between package managers not divergence.

Agreed. Either convergence, or refusal.

but the current ambiguous, erroneous middle-ground is harmful.

I still think that just throwing an error in the presence of the others' lockfile would require the least friction for existing projects.

[hsribei]

If you want npm to consider this I encourage you to open a discussion in the appropriate venue

Thank you, I've just signed up. There's a recent thread in this direction: https://npm.community/t/npm-install-should-warn-about-the-presence-of-yarn-lock-files/1822

My vote is for erroring out full-stop instead of just warning, so not sure if I should append to that thread of start a new one.

I would like to see convergence between package managers not divergence.

Same here. Though they are incompatible right now, so the divergence has already happened and is already causing friction to many users and projects.

Whatever convergence comes out of further talks amongst the teams is going to take time. When it does arrive, the proposed errors can be lifted and users can start switching and mixing package managers without a cinch.

Adding the errors now avoids further confusion and doesn't stop efforts in the direction of convergence.

I would recommend against some new "package-manager" field, instead recommending the existing engines stanza which already has prior art.

That sounds reasonable. I'm not attached to any particular method of detecting the presence of an incompatible package manager.

---

To add illustration, my use case in particular is gatsbyjs. The situation is the exact same as reported by @gaearon here:

To Create React App users, this is extremely confusing because their project gets created with Yarn (if Yarn exists on the system) but then they follow the docs of some library that tells them to npm install, and that blows away the whole tree.

To avoid this problem, even though gatsby uses yarn, it also adds a package-lock.json to default starters. But then users end up with both lock files and start seeing warnings. It's easy to just delete one of them, but it muddles up the onboarding experience.

[imsnif]

Thanks everyone for your input.

@jmorrell - https://github.com/yarnpkg/yarn/pull/5920 was released in 1.9.2 on Jul 25. Been a little over a month, I know it's not a lot of time (and surely not everyone has upgraded) - but do you maybe have some insights to share regarding the double-lockfile errors on Heroku since then?

[jmorrell]

@imsnif Thanks for the email reminder! Sorry for missing the update here.

Here is a graph of the number of apps that experienced this particular failure each week in 2018.

(I've redacted the y scale, but this represents 100's of developers and shows the trend)

1. The dip in July comes from an unrelated issue with S3

2. Looking at the trend after July 25, the warning seems to have had a substantial effect on the number of people experiencing this error. I am somewhat surprised by the magnitude of the shift.

[imsnif]

@jmorrell - this is really great! I'll admit I'm also quite surprised at this.

I think (on the yarn side) this issue can be considered resolved. Do you agree?

[Spongman]

i still think the advice of removing package-lock.json to fix the error is misleading - this will in general result in a different set of package versions being used, and will (again, in general) lead to incompatibilities and breakages. also, this advice is only useful for those who make decisions about which package manager a project is using. general contributors following this advice are also going to run in to issues.

IMO if a package-lock.json exists and yarn can't guarantee that the versions it installs are the same as those that npm would install, then it should fail with an error.

[imsnif]

Thanks again for your input @Spongman. I feel this has all been well discussed in the (very long) thread above us and I don't think it makes sense to restart it again. I think we all understand your position. Thanks for participating.

Unless @jmorrell tells me otherwise in the next week or so, I'm going to consider this done and close this issue.

[jmorrell]

@imsnif Taking a different track and looking at failures from September so far (Sep 1 - Sep 20), this is still the number 1 reason that Node builds on Heroku fail. It happens twice as much as the next most common known failure case: package.json being invalid JSON. Once again redacting exact numbers, here is how it stacks up to other issues.

I haven't seen any movement from npm on this, so I'll create a PR adding a warning there too and see it gets accepted.

I think (on the yarn side) this issue can be considered resolved. Do you agree?

I don't think this has been resolved for users, and the user experience still isn't great. But I can see how the numbers change once npm also gives feedback to a user hitting this case.

I'll leave it up to you whether you want to close this or make further changes. I'm happy to open a new issue in a few months if this is still a big problem for users.

[imsnif]

@jmorrell - seeing as this issue is attracting a lot of attention and the conversation keeps restarting and even repeating itself, I think it would be better to close it now with the information we have.

If, with npm providing a warning as well, we still see an issue, I'd definitely be willing to start the error discussion again (or look into other solutions). Please, do ping me personally in that case.

Thanks for bringing this up, contributing and making all these changes happen! I personally feel yarn is better after this issue.