Voting on the default behavior for Promise unhandled rejections

[mmarchini]

v15 Semver Major Cutoff: 2020-09-22

As discussed in the last TSC meeting, we should vote on this issue so we can remove the deprecation warning on v15, so the next step is to decide the voting format. My suggestion is:

• have the vote async instead of during a meeting so we can ensure folks have the opportunity to cast their votes
• private voting instead of voting on a public issue
  • this is a polarizing issue, might be good to try to avoid backlash on any TSC members because of their votes
  • public voting means folks will know the preliminary results before voting, which could influence their votes. Private voting doesn't have that issue
• TSC charter says we needs simple majority (winner needs 50%+ of votes) for a vote to pass, which works fine for yes/no votes, but since this is a multiple choices vote we might end up with no choices having more than 50% votes. My suggestion is to have a second vote if that happens with the two options with most votes

Any objections on using this format?

[mmarchini]

Also I'll share the raw responses and some extra analysis shortly

[mmarchini]

Added label for visibility, but preferably we should decide the format before the next meeting. If we haven't decided before the meeting we can discuss and decide the format during the meeting.

[benjamingr]

Just expressing my opinion, I am one person.

I think that given the survey, the older netflix survey, talking to users, the promises work from the 2018 and 2019 summits and usage from users as well as playing around with options I lean heavily towards throw as the behavior I'd like. My reasons are:

• It seems like it aligns with user expectation.
• It's one less things users have to remember since sync and async errors behave similarly.
• The thing I'm most concerned about is the fact the current behavior is easy to ignore since the warning is logged to stderr which makes it easy to swallow errors in production and cause cascading failures.
• I have seen a lot of bugs where having this default would have helped.
• It's very very easy to opt out of in multiple ways and a single line of code.

I am happy to elaborate on this further - feel free to reach out in person or in chat if you want to debate this.

I just wanted to say thank you to all the people involved in these discussions (namely Matteo, Ruben, Mary, Jordan, Julien, Gus and a few others as well as the people who helped at the start like Kris, Domenic, Stephan, Mark and Petka) for educating me on various bits and helping me formulate a much more informed opinion on the topic.

[mmarchini]

@benjamingr thanks for sharing. As I think we'll go to a vote anyway, it might be best for us to focus on the voting format now so we can have the vote sooner. We could try to go for consensus on one mode or the other, but chances are someone will object and we'll have delayed the vote for another week or so.

Also, if possible I'd like to keep this issue focused on the vote format, as I believe we have exhausted all possible arguments on both sides for that topic :)

[benjamingr]

Ok:

but since this is a multiple choices vote we might end up with no choices having more than 50% votes. My suggestion is to have a second vote if that happens with the two options with most votes

This sounds reasonable for me, I think it's a reasonable system arrow-theorem-wise given our alternatives.

[mmarchini]

Which tool do we use for private voting? Is there any tool the TSC has used in the past that worked for us, or should I reach out to OpenJSF to ask access to some tool?

edit: I'm reaching out pre-emptively so we won't delay it, but happy to use something already familiar if we have it

[MylesBorins]

In most non-controversial cases we've just voted publicly (for non-electoral votes). Often it is a yes / no majority vote type of situation. It seems like you are proposing that we vote on a matrix of possibilities, and that potentially we could have multiple votes... which to me seems like approval voting

https://en.wikipedia.org/wiki/Approval_voting

I believe that helios supports approval voting if we want to go in that direction... although depending on how many options we have we could use an issue + emoji or simply write in

[mmarchini]

It seems like you are proposing that we vote on a matrix of possibilities, and that potentially we could have multiple votes... which to me seems like approval voting

We have 5 possible resolutions for this issue, so it'll be a matrix. The second vote suggestion is to ensure the option chosen has support from 50%+ of TSC voters (if we get 50%+ in one option on the first round we won't need the second). I don't think what I suggested is the same thing as approval voting (not according to the wikipedia article), although approval voting could work as well?

FWIW this is what I'm proposing:

First round we'll be able to cast one vote for A, B, C, D or E. If A gets 25% valid votes, B gets 20%, C gets 14%, D gets 26% and E gets 15%. In this case, we'll have a second round between A and D.

I don't know how this voting system is called 😅, but I think I'm fine going with it or with approval voting.

In most non-controversial cases we've just voted publicly

This is a highly controversial and polarizing issue, so I still think having the vote in private would be ideal. I also think it's a good idea to remove the risk of early voters from influencing later votes (which could happen unintentionally in public voting because the votes are available before the results are shared). If we decide to have the vote public anyway, IMO we should lock the issue preemptively.

[benjamingr]

Just a comment on the name of the voting system - so I'm hiding it to keep discussion on the topic Mary requested.

I actually did several courses on voting systems (fun fact).

Some common rules are:

• Copeland (You choose the candidate who wins the most other characters in pairwise elections)
• Borda count (Each voter submits preferences over the alternatives, points are then given (from 0 to n-1 for each voter). Highest total wins. (Violates Condorcet winner))
• Single Transferable Vote (Candidate with the least points is eliminated at each round)
• Anti Plurality (Choose the least unliked candidate (the one the least agents placed last)).
• Sequential pairwise voting (Each time two candidates are placed against one another).

There are many others - the method you are describing is simply called Two-round or "runoff voting". It's fine for us.

[mhdawson]

I understand the suggestion for the vote being private due to the polarizing issue, but I'm wondering if it should be public to tsc members but private outside that group. Understanding the positions of other TSC members does factor into what I personally think we should do.

[mmarchini]

From TSC meeting today, we're going to try to reach consensus one more time in the TSC before going to a vote. Failing that we're going to a private runoff voting conducted through the TSC mailing list. I'll update the issue here as soon as we have an update on the consensus reaching attempt.

[mmarchini]

Consensus not reached, we started the voting process privately using OpaVote. Vote format will be San Francisco RCV. I didn't set a timeline but hopefully we'll get a result within the next few days (that's not a promise though)

[benjamingr]

that's not a promise though

thenable then? 😅

(sorry couldn't resist)

[mmarchini]

@nodejs/tsc remember to check your inboxes :)

[mmarchini]

We finshed our voting session. We used OpaVote to coordinate the voting and the vote format used was San Francisco RCV. We had four abenstees (out of 19). Results are:

OpaVote Election Results (https://www.OpaVote.com/)

VOTE: Default Behavior of Unhandled Rejections

There are 5 candidates competing for 1 seats. The number of voters is 15 and
there were 15 valid votes and 0 empty votes.

Counting votes using San Francisco RCV.

 R|strict     |throw      |warn       |warn-with-e|none       |Exhausted  
  |           |           |           |rror-code  |           |           
==========================================================================
 1|          1|         14|          0|          0|          0|          0
  |-----------------------------------------------------------------------
  | Count of first choices.
==========================================================================
 2|           |         15|           |           |           |          0
  |-----------------------------------------------------------------------
  | Count after eliminating strict, warn, warn-with-error-code, and none
  | and transferring votes. All losing candidates are eliminated.
  | Candidate throw is elected.

Winner is throw.

With that, we can move forward with the current open PR which changes the default behavior to throw.

[devsnek]

f

[DerekNonGeneric]

SGTM. Looking forward to it. :)

[ljharb]

That's a massive shame. Promise.reject() should not cause a JavaScript implementation to terminate.

[mmarchini]

@ljharb we understand not everyone will agree with the decision, but some default had to be choose and we decided on the one we believe will be the most beneficial for a majority of Node.js users. Note that none of the options would make everyone happy, that's just the nature of the issue.

If it turns out we release v15 and this decision becomes a huge pain across a large portion of the ecosystem (with evidence of that), we might reconsider it. But for now, throw is decided as the default.

[Jamesernator]

Note that none of the options would make everyone happy, that's just the nature of the issue.

It's not clear to me why Node.js restricted itself to a number of solutions that would almost certainly lead to a large number being unhappy, rather than creating a solution that would work as expected in the majority of cases.

A coherent solution would've provided a way to handle promises asynchronously without adding the promise.catch(() => {}) superfluous fluff.

As an example, it's not clear to me why throw-on-gc was not even included as one of the --unhandled-rejection rejection options, given that it accurately captures all promises that were never handled without breaking asynchronous handling.

As another example, a solution built on async_hooks (e.g. zones or similar), would've been another solution that gives clear boundaries for where asynchronous errors occur.

But instead we get something that does the bare minimum a solution even can, rather than providing a good out-of-the-box solution for handling async errors.

[devsnek]

with regard to async_hooks and zones and whatnot, we can always build on the decision that was made. with regard to throw-on-gc, it doesn't capture all promises, it captures all promises collected by the gc.

[mmarchini]

I don't remember exactly why throw-on-gc was left out back when the flag was added, it was decided during a session in the Vancouver Summit 2018 (maybe @BridgeAR remembers?), but I do remember the vast majority (if not unanimously consensus) on the session being against throw-on-gc as default. Also, as @devsnek mentioned, it doesn't capture all rejections that will never be handled, if a unhandled rejected promise is mistakenly retained in the gc tree it will never be collected.

It's not clear to me why Node.js restricted itself to a number of solutions that would almost certainly lead to a large number being unhappy, rather than creating a solution that would work as expected in the majority of cases.

These are the options we have after 4+ years of discussion. throw-on-gc was left out as explained above. If you have other suggestions, we're happy to hear.

As another example, a solution built on async_hooks (e.g. zones or similar), would've been another solution that gives clear boundaries for where asynchronous errors occur.

Do you have any specific ideas to share? Domenic zones proposal is highly unlikely to move forward on TC39 (even a highly simplified version that stripped out error handling completely is having trouble advancing stage), and so far there are no proposals (on TC39 or here) that don't fall on the problems we had with domains.

[Jamesernator]

it doesn't capture all promises, it captures all promises collected by the gc.

This is true, althugh the overwhelming majority of cases should still be caught. The only reason to hold a promise is to process it later. If a promise is held eternally but not processed, then that's an indicator of a memory leak which is not an issue unique to promises by any means.

As such it seems weird that the design for "unhandled rejection" is effectively extended to a specific case of memory leaks.

[mmarchini]

I think that's reasonable if instead of -on-gc we use WeakMaps and throw when the Promise is not retained by anything else. IIRC one of the concerns with -on-gc was the huge delay for V8 to GC Promises, but now we have WeakMaps so the implementation could be more feasible.

(I'm really sad that we lost the meeting notes for that session, otherwise we could revisit what were the main concerns with that proposal)

[Jamesernator]

Do you have any specific ideas to share? Domenic zones proposal is highly unlikely to move forward on TC39 (even a highly simplified version that stripped out error handling completely is having trouble advancing stage), and so far there are no proposals (on TC39 or here) that don't fall on the problems we had with domains.

While Zones was rejected by the TC39, the behaviour Node is adding is already doing something other than the suggested typical behaviour:

A typical implementation of HostPromiseRejectionTracker might try to notify developers of unhandled rejections, while also being careful to notify them if such previous notifications are later invalidated by new handlers being attached.

so it seems to me that it would as such be fine for Node to just create something similar to Zones without TC39 permission, given it already is providing divergent behaviour to other environments.

[ljharb]

Now that WeakRefs are stage 4, perhaps those could be used with a FinalizationRegistry for that approach?

[devsnek]

even promises which are not referenced by anything may be left in gc limbo. In the original pr I believe there are a few examples of this.

in any case, @mmarchini it seems like this can be closed?

[mmarchini]

If anyone is willing to go down the rabbit hole, here are the two PRs where throw on gc was proposed the first time:

https://github.com/nodejs/node/pull/20097 https://github.com/nodejs/node/pull/15126

Either way, even if we could get consensus or reach a decision again on throw-if-not-retained/throw-on-gc, someone has to implement it first. As I mentioned in my comment above, if throw turns out to be a huge pain for the ecosystem, we can re-evaluate the default behavior.

[mmarchini]

@devsnek I agree this should be closed since this issue was only for vote coordination, therefore discussion here is not really appropriate (folks who want to join the discussion might not be aware it's happening here)

@Jamesernator if you feel strong about this feel free to open an issue on https://github.com/nodejs/node to continue discussion.

[benjamingr]

Hey, just wanted to thank everyone involved and especially Mary for persevering and moving things forward when it was the difficult thing to do and required a lot of work.

So thanks Mary and to other people involved - looking forward to this finally being fixed.

[benjamingr]

Also worth mentioning that it's important we move swiftly on this and change the default.

We can (and should) explore "throw on GC" semantics in the future but I feel there needs to be a considerable amount of work before it gets presented to the TSC and I feel that the people who feel strongly about exploring it should likely do the (specification and research) work.