Promise.prototype.done

[medikoo]

With current design, the only way to process resolved value and be sure that unhandled exceptions are exposed, is via following

promise.then(function (value) {
  setTimeout(function () {
    // Process value
   }, 0);
}, function (err) {
  setTimeout(function () {
    // Handle error
   }, 0);
});

Which is poor solution for many reasons:

• Affects performance:
  • Creates promise objects we don't need
  • forces processing to further tick
• Unintuitive
• Verbose

In many promise implementations, there's already promise.done, which doesn't create any new promise, and invokes callbacks naturally (outside of try/catch clause).

As done cannot be implemented on top of available API, I think it's very important to include it in spec.

As side note, why do Promise.prototype.catch is specified? It can be very easily configured on top of the rest of the API, and it's usability is low.

[medikoo]

@ForbesLindesay

I'm also concerned that most people are not saying "We can add this later, for now lets just get something out the door" they're saying "This won't be needed once the development tools have caught up, lets never add it."

This is my big concern as well, and not specifying (or assuring existence of) done opens pandora's box of issues.

@erights sounds reasonable. I just can hope that indeed done will land in DOM and through that all will follow.

[ghost]

@wycats I'm curious. Can you provide an example of a programmer "intentionally trying to attach error handlers later"?

Thanks!

[juandopazo]

@zenparsing preloading stuff into a cache?

[erights]

@zenparsing http://research.google.com/pubs/pub40673.html The mint/bank deposit method returns a promise for whether the transfer of money succeeds or fails. Later or earlier, or even on a different machine, the escrow exchange agent does a Q.all on several of these promises, which register onReject handlers on them.

In my experience, this kind of delayed registration is quite common, and is one of the reason that promises are so convenient.

[domenic]

OK. So done is outside the scope of this repository; closing this issue. But to be clear, as @erights states, it's not off the table entirely for ES7 promises. We can put it in the bucket of things, like finally, that at least some people want ASAP as additions to the DOM promises subset this repository is designed to codify.

[spion]

I don't get it. Although late registration is common it still looks like a very special case to me. The vast majority of cases I've encountered don't involve sending promises over the wire and therefore don't seem to benefit from it.

On the other hand, it seems very common to forget the need to use .done at the end of a longer promise chain and end up with bugs that leave no traces and take forever to track.

Wouldn't it make more sense to have promise.undone() instead to indicate that the promise will be handled later? Or am I missing something obvious?

[medikoo]

@spion you're missing something obvious. Spec in current shape is missing a function that provides access to resolved value without any side effects, done is the one.

The best way to get is to use promises in real projects and not just theorize about it, but following above discussion should also be helpful.

[petkaantonov]

In your example, exceptions could have been exposed even if you had written:

promise.then(function (value) {
    // Process value
}, function (err) {
    // Handle error
});

Because it is possible for a promises library to detect that you don't have an error handler. So it is just common sense to report it or even throw it in the process.

Problem is that an error handler attachment could be delayed. And that's Spion's point: Why should everyday common usage be punished by requiring some obscure easy-to-forget method calls like .done() just because there is a possibility to attach late handlers? We should instead require those late handlers to call obscure methods, not the everyday average developers.

Also, you should not worry about promise object cost like that. You happily create 3 functions (1 of which is closure) regardless of path taken when your code is called which is already worth 5-15 promise objects depending on implementation so mentioning perf is not a strong point.

[wycats]

If you look upthread, you'll see that I proposed this a few times. The response argument, which I think I agree with after trying it, is that it's not always clear to a piece of code that receives a promise whether the error handler will be attached synchronously or asynchronously.

When you start building a system that abstracts asynchrony (especially one that forces asynchrony for consistency, like this one), it becomes very easy to lose track of whether some part of you code is attaching things synchronously or not. Users will then receive unexpected exceptions, without a clear understanding of the root cause.

Sadly, one's perspective on this problem changes over time from the beginning (when asynchronous attachment seems remote) to later on (when opting out of synchronous attachment seems like a huge burden).

[petkaantonov]

Users will then receive unexpected exceptions, without a clear understanding of the root cause.

The less extreme alternative I gave was just to report it, e.g. write the stack on stderr as possibly unhandled exception. You could then investigate it and either call undone() to signal that this has late handlers or fix the bug.

Sadly, one's perspective on this problem changes over time from the beginning (when asynchronous attachment seems remote) to later on (when opting out of synchronous attachment seems like a huge burden).

Yes, but the argument is still that it's better to force .undone() on people who do late attachment than to force done() on everyone, because late attachment is the less common case.

Unless one is mixing async and promise code instead of wrapping/promisification then as I see it, it requires deliberate setup to attach delayed handlers. I'd really like to see a code example where a delayed handler is attached inadvertently that doesn't involve remote promises or mixing paradigms.

[medikoo]

@petkaantonov It's not possible to fix side effects of then in bulletproof way in plain JS implementation, and in native implementation it requires specific optimizations and some custom error handling mechanism that no other API has seen so far.

What is obscure it's how we want to fix the issues that we introduce by not providing done.

Argument that then and done are too much for average developer's head is totally flawed:

• We already deal with similar concept in Array API (forEach and map), I don't recall anyone stating that's too confusing.
• If developer is not familiar with propagation of promises, he/she will have big problems with lone then already. There's when.js library which tries to provide just then and solve unhandled errors with some kind of monitor, and still there's a lot of misunderstanding, just browse the issues of this project to see what it turns into.

then and done are different methods that serve two different use cases, both are mandatory for promise implementation, and this statement I base on my two years experience of working with promises in real world projects.

[petkaantonov]

Well, let's just say we have very different definitions of "average developer" if you think they even use .forEach and .map. :smiley: I will check the when.js issue tracker but last time I checked the library doesn't provide long stack traces so I can see already how it would be less useful.

[ghost]

@spion Thank for bringing this up - I believe that your intuition is correct. I am preparing a post for es-discuss which will address this and other issues with the current Promise spec.

[spion]

I came up with an example that makes me wonder if I'm correct:

function fetchCachedOrRemote(item, remoteDataStorePromise) { 
    return cache.get(item).then(function(result) { 
        if (result) 
            return result; 
        else 
            return remoteDataStorePromise.then(function(store) { 
                return store.fetch(result).finally(cache.put); 
            });
    });
}

Not waiting for the remoteStore promise to resolve is not only convenient but also potentially much faster.

However, in this example its still beneficial to attach an error handler to the remoteStorePromise (e.g. to handle reconnection).

But I wasn't trying to make a case against .done, I was just looking for a simple example that demonstrates why attaching error handlers asynchronously is clearly more convenient. And I see how medikoo's argument is conceptually sound: .then creates a new promise, .done doesn't - so if you don't want a new promise, don't use .then

[erights]

a simple example that demonstrates why attaching error handlers asynchronously is clearly more convenient

See http://research.google.com/pubs/pub40673.html Even when used non-distributed, the patterns in this code do delayed registration.

[ghost]

@erights That code, although dense, is a great illustration of delayed registration patterns. I agree that delayed registration is an important feature, but the question is: What is the best way to support both delayed registration and also proper and timely identification of program errors?

Done does the job, but at the cost of (a) bifurcating the callback API, and (b) requiring that the user add done at the correct places.

More thoughts to follow...

[ghost]

A concrete proposal:

Conceptualize promises as nodes within a forest of trees. Root nodes are created with the promise constructor:

var root = new Promise(...);

Child nodes are created with then:

var child = p.then(...); // "child" is a child of "p"

Any leaf node which is rejected is clearly an unhandled error because there is no child for the error to flow into. The problem is: how do we know whether a given node is a leaf or an intermediate node whose children have not yet been added?

Under our current error handling model, all nodes are assumed to be non-leaves. That is, we assume that a child may be attached at any event in the future to any node. We are therefore unable to determine if any rejection is an unhandled error. Essentially, done provides us with a means of explicitly marking a node as a terminating leaf and thereby restoring that ability.

What if we reversed the assumption?

Let's assume that every node is a leaf, until a child has been added to it. More specifically:

• In PropagateToDerived(p), if p.[[Derived]] is empty and p.[[Reason]] is set, then queue a task to run immediately after the microtask queue is flushed which runs the following:
  • if p.[[Derived]] is empty and p.[[Reason]] is set, then propagate an unhandled exception in a platform-specific manner.

Additionally, add a new convenience method to the Promise prototype to explictly make a node a non-leaf:

Promise.prototype.delayed = function() {
  this.then(null, reason => reason);
  return this;
};

This method (which needs a better name) can be added at a later date. For the present it can be provided by a helper library. Coding patterns which depend upon delayed registration would need to invoke such a method.

[Nathan-Wall]

What is the best way to support both delayed registration and also proper and timely identification of program errors?

This answer seems easy and clear to me: Proper debugging tools for uncaught thens.

[ghost]

@Nathan-Wall That sounds cool, but:

• It presupposes an interactive debugging environment. That's fine for the web, but do you want to make such an assumption for, say, Node?
• If you are making heavy use of delayed registration, then presumably you'll have lots of things coming and going from your promise debugging tool. There will be noise that you'll have to filter (perhaps using some heuristic algorithm).
• I would think that ideally, "program errors", whether coming from sync or async patterns, should be reported in the same way. Otherwise, aren't we requiring the user to "switch views" constantly?

[wycats]

@zenparsing I proposed precisely this a while ago (with the worse, tongue-in-cheek name undone).

The problem is: who exactly in an abstraction should call it? The delayed attachment mostly happens as a side effect of a lot of async, rather than an intentional choice to delay attachment.

In other words, there isn't really a point where you think "I'm going to delay attachment now", it's something that just starts happening as the system grows.

[ghost]

@wycats What convinced you of that conclusion? Can you provide an example?

[medikoo]

@zenparsing

Done does the job, but at the cost of (a) bifurcating the callback API, and (b) requiring that the user add done at the correct places.

Don't you see that your proposal is much more obscure and makes things much worse?

Many say done and then is too much for dumb developer (which in my opinion is nonsense). I ask do you expect that your proposal would be much easier for developers to adapt?

Do you think developer will less likely forget to run delayed which now you require to add instead of done? and mind that done solves many more problems than one you're trying to solve in cumbersome way here.

There's null cost with done just benefits, and please realize done is not an add-on, if you want to look that way, it's then that is add-on to done not otherwise. done is the most atomic, generic one, that makes one thing right without any side effects. All API (including then) could be build on top of done, but not on then which does a bit more.

We're making attempts to reverse cost of then with crazy complicated, imperfect solutions and fail to realize that all that needs to be done is to provide more atomic done. It's not what future users of promises expect from us.

[petkaantonov]

delayed is not required in place of done. It is not required at all. It is only when you do something like pass around promises as if they were event emitters that you would potentially need to call delayed. I can agree that telling in such a situation whether async or sync attachment is happening is hard but I disagree that this situation is common in the first place - hell, all the examples everywhere (except the paper) use promises for their sync parallel, not as event emitters.

And even if you forget delayed all what happens is that you may get a message about potentially unhandled rejection. Not as bad as forgetting .done

[ghost]

@medikoo The thrust of your argument is that done / then are analogous to forEach / map. I understand what you're saying, but you're back-porting this analogy onto the design, rather than building it into the design. If you were going to build it into the design, you would choose different names. With a dual-callback API, then becomes an attractive nuisance because it's such a nice name. The API just wasn't designed for dual-callback.

Furthermore, my other point still stands: the user still has to remember to add done in the correct places in order for errors to show up in the error console.

@petkaantonov has it right. When you forget to add delayed, you get a nasty error real quick. This is how error handling ought to work. The programmer should not have to ask the environment for an error notification.

@wycats might still have a point though, we'll have to see. Although my first reaction is to say that as a general rule abstractions should be designed so that errors can be handled by something. : )

[ForbesLindesay]

The assumption that you get an error quickly when you forget to add delayed is not valid. When you forget to add delayed, it will probably work 95% of the time. The other 5% of the time, there will be a massive error because of the race condition you create. Since the promises are asynchronous, there's always a chance that the promise will take long enough to resolve that it gets watched before it resolves.

This issue is not to discuss the idea of early errors when you fail to handle errors in the same tick. If you want that, please open a separate issue. This issue is for deciding whether we want done. To be clear, making then throw an error that would crash an application is not an option we should be considering in this issue. There is a very important issue that does warrant discussion, and this issue is rapidly becoming long enough to need a summary.

[domenic]

If you want that, please open a separate issue.

Actually, please don't, or at least be aware that it will be closed immediately. Both done and any other proposed things anyone wants along these lines are outside of the scope of the TC39 September meeting consensus which this repo is meant to flesh out, and are off-topic to the real spec work going on here.

I would prefer that all this done rehashing take place elsewhere as well, but I don't feel like banning people, so whatever, if you guys want to use a closed issue as your own private mailing list that's fine I guess.

[ghost]

@ForbesLindesay Easy mate. We're just exploring - for kicks. : )

Regarding your argument, I think I understand what you're saying. Need a minute to think about it though...

[medikoo]

If you were going to build it into the design, you would choose different names.

That's possible, but it's too late to start over with blank paper, then and done names were already coined in many libraries, and we should stick to them.

user still has to remember to add done in the correct place

not to add, to use instead of then in the correct places

[ghost]

@ForbesLindesay Let's take a look at this.

Assume that I've unintentionally left out a delay on a promise which is supposed to be delayed. And let's assume that the resulting error was not revealed in testing due to environmental (async) concerns. What are the consequences?

1. For a web application, an error message will get logged to the console and window.onerror will get called. Otherwise the application will keep running as intended.
2. For a server application, it depends upon how the application handles uncaught errors. As a base case, the program will crash with a stack trace. More complicated applications might log the error before attempting some kind of graceful restart. Either way an issue will get logged and the bug will get fixed.

Since the system has no way of knowing that a rejection (which I forgot to delay) isn't in fact a program error leaving the program in an inconsistent state, then I think both consequences above are appropriate for their respective environments.

[ForbesLindesay]

@zenparsing read the entire discussion. Debugging tools will be able to catch up and trap unhandled rejections. They will also be able to report the bulk of the "never handled rejections" by hooking into the garbage collector at some point in the future. Non-deterministic bugs are by far the hardest things to fix. The way things currently are makes these bugs deterministic, which is a huge benefit.

@domenic I realize that it would be an issue that got quickly closed, but if that discussion had happened in a different thread, this could remain a useful record of the arguments for and against done. Sadly it's probably too late now :(

[ghost]

@ForbesLindesay I don't quite understand what you mean when you say that the current model makes "these" bugs deterministic. Can you elaborate?

[spion]

The current model always swallows these bugs when you forget to use done() but you're not using the return result of then(). In that way, its deterministic.

The delayed() model will swallow forgetting to use delayed() sometimes - only if the promise rejection happens before attaching the rejection handler.

function fetchCachedOrRemote(item, remoteDataStorePromise) { 
    // forgot to do remoteDataStorePromise = remoteDataStorePromise.delayed();
    return cache.get(item).then(function(result) { 
        if (result) 
            return result; 
        else 
            return remoteDataStorePromise.then(function(store) { 
                return store.fetch(result).finally(cache.put); 
            }); // finally handled via return value
    });
}

If I forget the first line in remoteDataStorePromise, there are two possibilities

1. cache.get responds faster than remoteDataStorePromise - no error
2. cache.get responds slower than remoteDataStorePromise - error.

Which in some situations may lead to bugs that can only be reproduced when the sun aligns correctly with the moon (e.g. the cache server is under super-heavy load, like in production)

Attaching to the GC is the only sure way to deal with this. Since there are no more references to the promise anywhere in the program, no further handlers can be attached. Therefore the runtime can report the error (e.g. log it or throw it in node.js). Perfect.

[ghost]

@spion Thanks - this is just what we need.

First, I hardly think swallowing of program errors can be considered helpful, deterministic or not. So we can leave that argument behind.

Second, I wouldn't be so sure that using the GC is "perfect". First of all, the garbage collector does not give you any guarantees about whether a particular object will be collected. (Correct me if I'm wrong, please.) Furthermore, it is quite trivial to construct a case where the GC does not collect a rejected promise which represents a program error: simply assign the promise to a long-lived variable.

To put it more concretely, given this (note the misspelling):

function F(someVar) { G(someVr); }

why should this:

fetchURL("/some/url").then(F);

have different error handling semantics than this:

var p = fetchURL("/some/url").then(F);

Before I go into your code example, let me point out that under the current model detecting an unhandled program error [edit: within a finite time of the error ; )] is undecidable (in the full Turing sense of the word)*. This is a vast departure from stack-based error handling and it deserves exploration.

*I think this is a valid reduction: https://gist.github.com/zenparsing/7030884

[ghost]

Let's take a look at @spion 's code (reformatted somewhat):

function fetchCachedOrRemote(item, remoteDataStorePromise) { 
    return cache.get(item).then(result => { 
        if (result) return result;

        return remoteDataStorePromise
          .then(store => store.fetch(result))
          .finally(cache.put);
    });
}

It seems to me that implicit in this code is the assumption that fetchCachedOrRemote is not "responsible" for any errors that may be represented by remoteDataStorePromise. It uses that promise, but it does not "own" it.

Also, the "responsibility" for any other promises generated in this function are transferred out to the caller.

Do you agree?

[spion]

Yes I agree - my example is lacking. Whoever created the remoteDataStore promise is ultimately responsible of handling its error conditions (e.g. retrying the connection, perhaps). Therefore my example only shows that late registration is beneficial in general, but not that there is a need for late registration of error handlers

Another important thing that I noticed is that the example in the paper posted by @erights is using promise cancellation for control flow while @zenparsing and I are thinking of handling error conditions / recovery - two very different things with different handling responsibilities.

I still haven't managed to come up with an example where delayed registration for error handling/recovery is obviously useful as opposed to early error handling. I'm currently searching for one :) If anyone with more experience using promises can come up with a simple one, I'd be really glad.

However, I now understand that .delayed() / .undone() will almost definitely not help - it will only result with hard to reproduce bugs.

@domenic - what would be the best place to continue this discussion - perhaps es-discuss? I feel like we're abusing this issue as well as possibly annoying the people subscribed to it...

[ghost]

However, I now understand that .delayed() / .undone() will almost definitely not help - it will only result with hard to reproduce bugs.

I agree. As proposed, delayed does nothing but swallow errors. Let's drop it from the discussion. It doesn't bear on our present line of inquiry, anyway.

[ghost]

Clearly decidable error handling (as I proposed above) is no less powerful than undecidable error handling - one could simply "delay" any promises that are going to be handled later. But as we've seen, that isn't a very good programming strategy.

Our task now is to come up with a reliable pattern for handling errors within the context of a decidable error handling model.

First, some terminology:

• Up the call stack: In the direction of the caller of the current function.
• Responsibility: For a function, the state of having the duty to handle a rejection.
• Handled promise: A promise which is the result of a direct call to then whose error callback converts all rejections into values.
• Unhandled promise: All promises which are not handled.

I propose the following strategy:

Responsibility flows in one direction only: up the call stack.

Or, more precisely:

Unhandled promises that flow up the call stack into a function invocation F must be either:

• pipelined (resulting in a new promise to which this rule is recursively applied),
• handled, or
• returned up the call stack to the caller of F

before the F is cleared from the call stack.

Can we think of any use cases that this rule leaves out?

[ForbesLindesay]

@zenparsing or the promise must be stored in a variable somewhere, that can be later referenced and handled. You're trying to solve the halting problem (which is known to be impossible). The garbage collector does a pretty decent job. We can't "crash the app" when the unhandled rejection is noticed by the garbage collector, but we can log it and alert the developer to the error.

The reasoning for done has only a little to do with error handling, and far more to do with how purely your intent can be specified.

[medikoo]

The reasoning for done has only a little to do with error handling, and far more to do with how purely your intent can be specified.

Actually It has a lot to do with error handling as well. It's the only function that allows us to process resolved value outside of promise specific error handling mechanism. I see it as no less important feature.

Mind also that API in current shape doesn't provide any way to expose unhandled errors (they'll be silent). Therefore it cannot be perceived as valid for a complete implementation. Adding done would solve that shortcoming.

[ghost]

@ForbesLindesay I'm certainly not trying to solve the halting problem! As I've laid out, the current error handling model is undecidable. That is, the problem of determining whether a particular error has a handler is undecidable. In my opinion, this is a mistake.

On the other hand, under the proposal at https://github.com/domenic/promises-unwrapping/issues/19#issuecomment-26254979 the problem is decidable.

The question becomes, do we lose anything with decidable error handling? I don't think we do, but I'm trying to work my way through that question.

[ForbesLindesay]

You can either:

1. Loose deterministic behavior (this isn't going to be accepted)
2. Throw/Log on garbage collected unhandled rejections (this won't catch every possible case, but gets most of the way)
3. Add a .done method (this allows pure expression of intent, will catch any case where it's added, but can be accidentally forgotten).
4. Solve the halting problem (this is the only thing that catches everything. It's actually the only thing that catches more than the garbage collector without people explicitly adding then.)

1 is a non-starter. 2 and 3 are both simple and extremely useful. 4 is proven to be impossible.

[ghost]

The question I'm pursuing is, can we "do" promises in such a way that error handling is both deterministic (to use your terminology) and decidable.

If we adhere to the rule at https://github.com/domenic/promises-unwrapping/issues/19#issuecomment-26606667 , do you agree that error handling is deterministic? Or am I misinterpreting your meaning?

[Nathan-Wall]

@ForbesLindesay, there's another version of 2 which is to log unhandled rejections and unlog them when they get handled, so that it's not dependent on the garbage collector... maybe the debug tools could permit you to filter by rejections that had been garbage collected for a better view, but non-collected unhandled rejections could be made visible.

[petkaantonov]

How is .done() (without arguments) not just a .catch(thrower).. so just do:

Promise.prototype.done = function() {
    this.catch(thrower);
};

[juandopazo]

If your thrower function is making sure the exception gets thrown, the only difference is that done shouldn't return a new promise becaus it's not a mapping operation.

[petkaantonov]

Yes I removed the return

[medikoo]

@petkaantonov catch is just sugar over then, and you should also explain what's thrower in your sample, as I assume it's nothing provided by the standard.

promise.done(processValue) vs promise.then(processValue).catch(thrower), latter ugly as hell, produces 2 obsolete promises (not needed for anything), and error is exposed next tick later than in case of done.

[petkaantonov]

My point was that .done without arguments is trivially implementable even if it's not provided out of the box. Also what difference does it make if your server crashes with a 3 micro second "delay"?

function thrower(err) {
    process.nextTick(function(){
        throw err;
    });
}

[medikoo]

@petkaantonov Issues are

1. Cleanness of code, need to put catch(thrower) at end of each promise chain. We really don't want to do that, and mind we need to provide thrower ourselves.
2. Obsolete promises created for no valid reason, that affect performance (speaking to performance purist ;-)
3. Next tick error resolution is just another dirty aspect of it.

[petkaantonov]

You can just put .done() since you can just add it to prototype (aren't they a wonderful thing). Although .catch(thrower) actually says what it does but whatever.

A chain of promises requires creation of closures due to unutilized this so the created promise object is literally a drop in the ocean.