SubscriptionObserver should never throw, just like Promise

[jhusain]

Observables, like Promises, can be either multicast or unicast. This is an implementation detail, should not be observable. In other words, it should be possible to switch an Observable from unicast to multicast without changing any code in any of the Observers of that Observable.

Given that design constraint it seems necessary that SubscriptionObservers, like Promises, must swallow errors. Allowing errors thrown from Observers to propagate can leak details of whether an Observable is unicast or multicast. To demonstrate this issue, consider the following Subject implementation:

class Subject extends Observable {
  constructor() {
    super(observer => {
      this._observers.add(observer);
    });
    this._observers = new Set();
    return () => this._observers.delete(observer);
  }

  _multicast(msg, value) {
    const observers = Array.from(this._observers);
    for(let observer of observers) {
      observer[msg](value);
    }
    return undefined;
  }

  next(value) {
    return this._multicast("next", value);
  }
  error(error) {
    return this._multicast("error", error);
  }
  complete(value) {
    return this._multicast("complete", value);
  }
}

Note in the code above that if an observer throws from either next, error, or complete, then none of the remaining observers receive their notifications. Of course, we can guard against this by surrounding each notification in a catch block, capturing any error, and rethrowing it later.

  _multicast(msg, value) {
    const observers = Array.from(this._observers);
    let error;
    for(let observer of observers) {
      try {
        observer[msg](value);
      }
      catch(e) {
        error = e;
      }
    }

    if (error) {
      throw error;
    }
    return undefined;
  }

Unforunately this only works if there is only one error. What if multiple observers error? We could aggregate all observer errors into a CompositeError like so:

  _multicast(msg, value) {
    const observers = Array.from(this._observers);
    let errors = [];
    for(let observer of observers) {
      try {
        observer[msg](value);
      }
      catch(e) {
        errors.push(e);
      }
    }

    if (errors.length) {
      throw new CompositeError(errors);
    }
    return undefined;
  }

Unfortunately now we have a new leak in the abstraction: a new error type which is only thrown from multi-cast Observables. This means that code that captures a particular type of error may fail when an Observable is switched from unicast to multicast:

try {
  observable.subscribe({ next(v) { throw TypeError })
} catch(e) {
  // worked before when observable was unicast,
  // but when switched to multicast a CompositeError thrown instead
  if (e instanceof MyCustomError) {
    //...
  }
}

Once again we can see that the interface changes when moving from a uni-cast to multi-cast Observable, and the implementation details leak out.

The only way I can see to cleanly abstract over multi-cast and uni-cast Observables is to never mix push and pull notification. I propose the following simple rule: subscribe never throws. All errors are caught, and if there is no method to receive the push notification that the error occurred, the error is swallowed.

[zenparsing]

@blesh See here for the motivating use case.

But I think it's probably the right thing to do.

It won't work ; )

[benlesh]

@zenparsing I think the motivating example might be relying on the abstraction leak. This forEach will stop if something farther up the call stack from listener.next throws.

    req.publish = event => {
      listeners.forEach(listener => listener.next(event));
    };

It seems like you're saying that the desired behavior if one of your subscriptions down stream throws is to abruptly stop notifying all other listeners and remove them all from the list of listeners. If I'm a consumer, it doesn't make sense to stop sending me values because some other consumer errored consuming the values.

Unless I'm misunderstanding. Can you possibly give the use-case in terms of desired functionality and not in terms of code?

[zenparsing]

I'll follow up in more depth tomorrow, but the code is the desired functionally. On the server, I want an error town from any listener to immediately unwind the stack until it hits my exception handler, which will then send the error through the Express error handling router.

[benlesh]

There are a few things about this example that confuse me. For one thing the events Observable is created, but doesn't seem to be used anywhere. It looks like the goal here it to set up a multicast observable. I'm just not sure what it's used for.

It seems like you could handle this problem with a simple callback. The advantage being that you could then notify for each error on every listener, rather that ceasing to try to notify the listeners at the first sign of trouble. I mean, I think there are a lot of ways to compose this.

function eventHubMiddleware() {
  let listeners = [];

  let events = new Observable(sink => {
    listeners.push(sink);
    return () => {
      let index = listeners.indexOf(sink);
      if (index >= 0) {
        listeners.splice(index, 1);
      }
    };
  });

  return (req, res, next) => {
    req.publish = (event, errorCallback) => {
      listeners.forEach(listener => {
         try { listener.next(event)); } catch (err) { errorCallback(err); }
      });
    };
    next();
  };
}

app.use('a', (req, res, next) => {
   req.publish(new SomeEvent(), next);
});

[benjamingr]

If we use hooks similar to unhandledRejection and rejectionHandled then this isn't really an issue.

If observable subscriptions are going to trap errors - we might as well make .subscribe I'm interested if we can get .subscribe to return a promise-like and reuse all the existing instrumentation.

[zenparsing]

@blesh Right, you're proving my point: this change makes Observable unsuitable for this use case. You can't implement Node's EventEmitter with it either.

[jhusain]

Hey @zenparsing,

Thanks for clarifying the use case you're concerned about. Have been preparing a reply, but will wait for your expanded response. Just wanted to the add the following datapoint to the discussion: EventTarget does not propagate errors to the dispatcher, even when the events are dispatched synchronously. Errors appear to be sent to HostRejectionErrors, because they are still logged to the console.

The committee insists that EventTarget must be able to be built on Observable. Given that error swallowing matches EventTarget behavior, error swallowing can meet the committee bar. Allowing errors to propagate (as @zenparsing suggests) also meets this bar, because it leaves what happens up to the implementation. This is not a strong argument against or for either proposal, but it would've been a disqualifier for error swallowing if EventTarget had allowed an error to propagate to the dispatcher. Just wanted to add this datapoint, because it is important.

[zenparsing]

@jhusain I don't think I need to provide any more detail; @blesh has helped me clarify my argument.

I've always thought it should be possible to implement both EventTarget and Node's EventEmitter in a straightforward way using Observable. With the current design, this is possible. With the swallowing behavior, it is not.

[jhusain]

@zenparsing Thanks for clarifying the use case you're concerned about. Node APIs do not make decisions based on the completion value (abrupt or otherwise) of callback notifications. In situations where a Node push API wants to handle an error from the consumer, the producer provides an error callback for the consumer to invoke. The web platform did effectively the same thing with ServiceWorkers, which expect consumers to set a Promise on the event object to communicate success/failure back to the producer. This is also the solution I would advocate for your use case.

observable.subscribe(e => {
  e.response = (async function() {
    // handling code in here may be asynchronous
  }());
});

Node APIs do not swallow errors (including EventEmitter), but that's not to enable dispatchers to catch them and make decisions. The current guidance in Node is to let all errors thrown from callbacks propagate uncaught and take down the process. Why? The guidance in Node is to use the throw channel for "unexpected" errors. The definition of an unexpected error is somewhat fluid, but the idea is that it is a programmer error - like a null reference error. The thinking is that these errors can't really be handled and therefore should never be caught, causing them to propagate and tear down the process as soon as possible. This approach gives Node servers with two benefits:

1. Precludes the possibility that the server will process a request after being left in an invalid state due to an unexpected error.
2. If an error is allowed to propagate without ever being caught, it is possible to get a detailed core dump which provides a lot of observability with respect to the state of system when the error occurred.

Conversely "expected errors" are those errors that the developer anticipates and expects to be able to handle gracefully. A good example of an "expected" error would be getting a 500 from a downstream service. In Node, the guidance is that consumers should communicate expected errors via callback.

There's a good reason why Node servers use callbacks to communicate errors that can be handled: forcing consumers to synchronously handle notifications makes it impossible to subsequently move some of the handling logic "off-box" to balance load. This happens all the time, as systems tend to become more distributed in response to increased complexity and load. Today you may have one Node server and a SQL database, but tomorrow you may have a microservice architecture and 14 different document stores. Forcing consumers to synchronously handle notifications makes this transition impossible, which is an excellent example of the refactoring hazard I mentioned earlier in the thread.

It's well-known that the Node guidance to use throw for unexpected errors and callbacks for expected errors is incompatible with Promise error swallowing. People on the committee and the Chrome team are actively thinking about how to add hooks to the platform and ecosystem to provide better safety and visibility to Node developers in the context of Promise error swallowing. It is precisely these ecosystem hooks that Observable will benefit from if we harmonize the way Promise and Observable handle errors.

@zenparsing To summarize, using thrown exceptions to communicate handler failures to publishers is not a pattern that exists on the web, and it's not recommended in Node. Both platforms use a more robust solution: push-based feedback. Under the circumstances do you feel that supporting this particular implementation of producer feedback is compelling enough to justify Observable having different error handling behavior than Promises?

[jhusain]

-- some edits made to fix code bugs and for clarity --

@zenparsing You presented a use case in which a producer needs to be able to respond to an error encountered while a consumer handles a notification. I pointed out that this use case was not precluded by error swallowing, because a callback could be provided to the consumer to signal errors back to the producer.

Now you're expressing an altogether more narrow concern around implementation details: "it should be possible to implement both EventTarget and Node's EventEmitter in a straightforward way using Observable." Based on my recollection, this is the first time you've applied this bar to this proposal (with respect to EventEmitter). I would've appreciated it if this constraint had been made explicit earlier in the process. My apologies if you've communicated this to me via some side channel and I've forgotten - I'm not known for having a stellar memory :). However the additional qualifier of "straightforward" is particularly concerning because it potentially applies a much higher and subjective bar to the proposal.

It is certainly possible to implement EventEmitter on Observable with error swallowing:

class EventEmitter {
    constructor() {
        this._observers = new Set();
        this._subject = new Observable(observer => {
            this._observers.add(observer);
            return () => this._observers.delete(observer);
        });
    }
    on(eventName, callback) {
        var self = this;
        self._subject
            subscribe({
                next({name, value}) {
                    if (name === eventName) {
                        try {
                            callback(value);
                        }
                        catch(error) {
                            self._error = error;
                        }
                    }
            });
    }
    emit(name, value) {
        try {
            for(let observer in Array.from(this._observers)) {
                observer.next({name, value});
                if (self._error) {
                    throw self._error;
                }
            }
        }
        finally {
            self._error = undefined;
        }
    }
}

@zenparsing This implementation may not meet someone's subjective bar of what is "straightforward", but I respectfully suggest that this is an unreasonable basis on which to block a proposal.

Demanding that Observables don't swallow errors just because EventEmitter does not swallow them is just a rehash of the Promise error swallowing debate by proxy. @zenparsing If we were to uniformly apply your constraint then Promises would not have been introduced into the language, because Promises swallow errors and Node callbacks do not. I'm still very interested in a use case that convincingly demonstrates why Observable and Promise should differ in their error handling behavior.

[benjamingr]

In situations where a Node push API wants to handle an error from the consumer, the producer provides an error callback for the consumer to invoke.

Can you give some examples of that?

two channel talk

Well, a key aspect of operational vs programmer errors is that you can't determine if an error is operational or programmer on the producer side (a classic example is a JSON.parse on invalid database data). The sync/async -> programmer/operational translation doesn't hold water. When we started nodejs/promises we didn't all really understand it yet.

The core dump issue is a big and real problem with promises - I think actual asynchronous stack traces (like in C#, and unlike what we currently have in Chrome) would be a key milestone for promises in core. Core dumps are a real issue and although a lot of alternatives have been suggested to address core dumps with promises in core - the two are pretty incompatible so far. Then again - the crowd who uses core-dumps is pretty small minority in node.

I also like to point out that in practice people and companies using Node don't abort on exceptions. It is common to core dump on the first worker that has an exception and then to handle the rest. The DoS potential otherwise is huge.

Exceptions with callbacks propagate in a very ugly way where an intermediate layer between two of your APIs can leak which is a huge problem. It's why domains failed and why it is recommended to not handle synchronous errors you-re not sure about - you're likely creating a leak. When control flows normally (with async functions for instance) this is not an issue. In my experience with Rx and Node this is not really an issue with Rx since control is a lot like with promises and not like with callbacks.

(I'll gladly find nodejs/node nodejs/post-mortem and nodejs/promises references for all of the above).

---

On another note, it would be great if we stopped using the phrase "promises swallow errors" since that's clearly not the case. If you throw an error in a promise and no one catches it - node will throw, we're only not throwing because we're waiting for a v8 hook - and until then we're still warning in the console.

[jhusain]

In situations where a Node push API wants to handle an error from the consumer, the producer provides an error callback for the consumer to invoke.

Can you give some examples of that?

Sorry this was just dead wrong. I had misremembered that the error parameter passed to node callbacks was a callback itself, rather than a value. As far as I know there is no example in Node of a producer passing an error callback as part a notification to allow communicating back to a producer. This doesn't change the fact that Node APIs don't make decisions based on the completion value (abrupt or otherwise) of callbacks (AFAIK).

On another note, it would be great if we stopped using the phrase "promises swallow errors" since that's clearly not the case. If you throw an error in a promise and no one catches it - node will throw, we're only not throwing because we're waiting for a v8 hook - and until then we're still warning in the console.

From now on I'll just say that Observables should "catch" errors thrown from the consumers, like Promises.

[zenparsing]

This implementation may not meet someone's subjective bar of what is "straightforward", but I respectfully suggest that this is an unreasonable basis on which to block a proposal.

Thanks for the code example. No, I would not call that straightforward, I would call it a hack and I hope anyone attempting to use a built-in language provided API for such a simple use case would agree. 😄

That said, I apologize for being too vague in my requirement concerning EventEmitter. Of course it is possible to implement EventEmitter using a swallowing Observable. But we want more than that. We want to be able to retrofit EventEmitter so that it can vend Observables (just like we want to do with EventTarget - and I'm not sure why we would want it for EventTarget and not for EventEmitter).

Something like this:

EventEmitter.prototype.on = function(eventName, handler) {
  if (!handler) {
    return this._subject
      .filter(({ name }) => name === eventName)
      .map(({ name, data }) => data);
  }
  // The normal stuff
};

The trouble here is that the Observables vended from on("type") will have different behavior than the on callbacks:

// Errors will propagate
emitter.on('foo', x => { throw new Error() });
// Errors won't propagate
emitter.on('foo').subscribe(x => { throw new Error() });

You'd have to re-implement subscribe to make them agree.

Again, my point remains the same: swallowing-based Observables are not useful for implementing observables with EventEmitter-like semantics. Responses which cast those use cases as "bad programming" don't feel genuine to me, nor does the attempt to retroactively state that those use cases are not appropriate for an Observable API. There was nothing wrong with supporting those use cases before, but there is now? Why?

Demanding that Observables don't swallow errors just because EventEmitter does not swallow them is just a rehash of the Promise error swallowing debate by proxy

I object strongly to this characterization of my position. As I have said elsewhere, Promise error routing has very little correspondence with observable error routing. They are completely different.

For promises, the consumer can always catch the errors as long as you have a reference to the promise which represents the completion. But in the following, notice how neither the producer nor the consumer has any opportunity to catch the error:

subject.subscribe({
  next(v) { throw new Error() },
  error(x) { console.log('Good citizen, right? Nope!') },
});

subject.next(x);

I do not believe it is wise to introduce a primitive which will make it impossible for anyone to handle exceptions locally.

Now, of course there are some cases where this information hiding and breaking of the traditional exception propagation rules makes sense; EventTarget is one. Fortunately, it is quite easy with the current spec to create an Observable with EventTarget information-hiding semantics.

So for me, it comes back to this:

• Current spec: supports both EventTarget and EventEmitter-style propagation in a natural way
• Proposed changes: EventEmitter-style propagation is impossible

In order to accept such a tightening of scope, there would have to be a very good reason and I still don't see it. Does it come back to the multicast/unicast thing? The "abstraction leak" still seems very edge-casey and questionable. Doesn't it come down to exceptions thrown from subscribe?

Can a user just write a subject such that CompositeErrors aren't thrown from subscribe?

[ljharb]

Would subject in your case not be able to provide a rejected Promise, like subject.forEach().catch? Or am I missing something fundamental about the relationship between an Observable and the Promise for its final value (or error state).

[zenparsing]

@ljharb

Like any Observable, subject.subscribe would return either a subscription (the current spec) or undefined (the cancel-token-based spec), but not a promise.

The "promise for it's final value" only comes into play when using forEach. A promise is returned from forEach, and presumably this work work just as you'd expect:

subject
  .forEach(() => { throw new Error() })
  .catch(error => console.log(error));

But that doesn't help subscribe.

[ljharb]

I'm asking about:

const sentinel = new Error();
subject.subscribe({
  next(v) { throw sentinel },
  error(x) { console.log('Good citizen, right? Nope!') },
});

subject.next(x);

subject.forEach().catch(x => assert(x === sentinel));

Meaning, the promise rejection would give you the opportunity to respond to the error thrown in the next method.

[benjamingr]

By the way, I still don't understand why this is an issue since a .subscribe can't emit a value in a way that's both multicast and synchronous. Multicast is always asynchronous for at least all but one consumer.

try {
  observable.subscribe({ next(v) { throw TypeError });
} catch(e) {
  // this will _only_ enter if observable is both unicast and synchronous, if it is multicast
  // then at most one subscriber will enter this `.catch`.
}

Therefor, I don't understand what catching inside subscribe does for us. We will never have a case where we need to emit an AggregateError.

I think HostReportErrors is the obvious choice as discussed here, where each subscriber reports its own error. The code doesn't work in an unrecoverable way anyway if we got to HostReportErrors.

---

@zenparsing

// Errors will propagate
emitter.on('foo', x => { throw new Error() });
// Errors won't propagate
emitter.on('foo').subscribe(x => { throw new Error() });

If it's anything like promises - the error propagates but not in the synchronous exceptions channel.

[zenparsing]

@ljharb Sorry for not being a little more clear with the examples. The subject in this case would be a typical Rx-style "Subject", which is a multicast observable with a "next" method. The "next" method synchronously sends a next value to all of the observers.

Notes added below:

const sentinel = new Error();

// Observer A is registered:
subject.subscribe({
  next(v) { throw sentinel },
  error(x) { console.log('Good citizen, right? Nope!') },
});

// This sends a value "x" to Observer A
subject.next(x);

// Observer B is registered:
subject.forEach(() => {}).catch(x => assert(x === sentinel));

So at the time that the message is sent, observer B has not been registered yet. Also, for a typical Subject we would not allow information from observer A to propagate to observer B.

Hope that clears it up a little!

[zenparsing]

For reference, to create a one-way observable that sends errors to HostReportErrors using the current spec, you can do:

class OneWayObserver {
  constructor(observer) { this._observer = observer }
  next(x) { this._send("next", x) }
  error(x) { this._send("error", x) }
  complete(x) { this._send("complete", x) }
  _send(method, x) {
    try { this._observer[method](x) }
    catch (err) { HostReportErrors(err) }
  }
}

class OneWayObservable extends Observable {
  constructor(subscriber) {
    super(observer => subscriber(new OneWayObserver(observer)));
  }
}

[jhusain]

This thread is getting hard to follow, so I'll try and summarize the arguments:

It seems we have the following open question: should the Observable implementation be able to receive errors thrown from Observer notifications?

This question is really a proxy for a more general design question: should we...

1. base Observables on generators, which can receive feedback values from consumers and act upon them.
2. base Observables on the iteration protocol in which the producer does not receive feedback from the consumer.

The proposed implementation of option #1 would be for SubscriptionObserver to return the return value of input Observer to the Observable implementation.

The proposed implementation of option #2 would be for the SubscriptionObserver would always return undefined when notified, even if the underlying Observer notification code threw or returned a different value. Furthermore the complete () notification would no longer accept a value.

Here are some of the arguments made in support of modeling Observables on generators:

• strict superset of semantics means that it's possible to meet all the same use cases as an Observable modeled on the iteration protocol.
• Bidirectional communication is useful sometimes (ex. a producer wants to know if a consumer failed to process a notification)
• It would be more ergonomic to implement EventEmitter on Observable if Observable also allows errors thrown from callbacks to propagate.
• If EventEmitter is to vend Observables, then EventEmitter should have the same semantics as an Observable.

Here are the arguments in support of an Observable based strictly on iteration semantics:

• There are no known instances of a bidirectional Observer pattern in either Node or on the web.
• bi-directional communication is still possible within the context of a unidirectional Observer pattern. On the web this is accomplished via the mutation of data sent to the notification (ex. e.preventDefault(), e.promise = async function() { }.
• Supporting bidirectional communication makes the common use case of unidirectional observation less ergonomic because Implementers have to explicitly catch errors thrown from Observers, and consumers to explicitly try/catch calls to subscribe just in case subscribe runs synchronously.
• EventEmitter is not a use case for bi-directional communication, because the error is never caught by the EventEmitter. The decision to let errors propagate is consistent with a blanket policy in node of not catching errors thrown from callbacks to ensure that the process is torn down. The TC-39 chose not to support this pattern in the language's push primitives when it decided that Promises should catch errors.
• It is still possible (though less ergonomic) to implement an EventEmitter that throws errors thrown from callbacks using an unidirectional Observable.
• There is no reasonable developer expectation that if an EventEmitter vends an Observable then the two interfaces must have identical semantics. By the same logic Node APIs which accept callbacks today cannot become thenables in the future, because Promises catch errors and Node callbacks do not.

I have some additional perspectives on these proposals which I will share, but I just wanted to see if I could summarize the arguments. @zenparsing can you add any data points I've missed and make corrections if necessary? - thanks

[zenparsing]

@jhusain Thanks for the summary, a few replies:

base Observables on the iteration protocol in which the producer does not receive feedback from the consumer

I think this statement should be clarified. The Iterator interface does not preclude accepting an argument, as explained here. As such, we should be clear that we are referring to the "for-of" protocol specifically.

There are no known instances of a bidirectional Observer pattern in either Node or on the web.

Strictly speaking this is false. I have a production application which uses some aspects of bidirectional communication. You could say it is uncommon.

Supporting bidirectional communication makes the common use case of unidirectional observation less ergonomic

I think "common case", at least as it applies to exceptions, is debatable since current implementations generally allow exceptions to propagate. EventTarget seems to be the exception rather than the rule here.

EventEmitter is not a use case for bi-directional communication, because the error is never caught by the EventEmitter...

This may be true for Node's core emitters, but users can create their own emitters which catch errors on emit, so I think the statement in it's general form is false. Also, please, no appeals to committee authority 😉 .

By the same logic Node APIs which accept callbacks today cannot become thenables in the future, because Promises catch errors and Node callbacks do not.

I think this statement a stretch. It's true that in the callback/promise scenario the producer loses the ability to see errors. But the consumer always has the ability to see them through .catch(...) chaining. With one-way observables, neither the producer nor the consumer can see the errors. The error can only be handled "globally" with something like process.uncaughtException.

You've left out an essential (perhaps the essential) argument for bidirectional communication: one-way observables would be the first built-in JS feature I'm aware of that doesn't allow "local" exception handling. By "local" I mean something like the following:

For any function call, an error arising from the execution of that function (represented as either an exception or an eventually-rejected promise returned from the function) can be handled by the function caller.

While it makes sense in some specific cases, I don't think we want to introduce a fundamental building block into the language which violates this rule.

[benlesh]

Strictly speaking this is false. I have a production application which uses some aspects of bidirectional communication. You could say it is uncommon.

@zenparsing do you have anything I can look at on Github that requires this use case? I've never seen anything like it, and I'm having a hard time wrapping my head around why it would be a requirement.

I think "common case", at least as it applies to exceptions, is debatable since current implementations generally allow exceptions to propagate.

This might just be because of an optimistic design for existing implementations like RxJS. I personally would like to do away with the current behavior in favor of error swallowing because of really spooky issues I've seen in production involving multicast observables that are handed to third parties throwing and killing everyone else's subscriptions. I've seen this problem more than once, and it's not something that's easy to explain to users. It requires a deep understanding of what's going on inside the library, and the workaround requires further explanation and comes with heavy performance penalties.

[zenparsing]

@blesh

I've never seen anything like it, and I'm having a hard time wrapping my head around why it would be a requirement.

Just focus on the error propagation part (which is also bi-directional). The specific use case is here. More generally, it comes up anytime one might want to do this:

try {
  subject.next(nextValue);
} catch (err) {
  // Do something reasonable
}

I personally would like to do away with the current behavior in favor of error swallowing because of really spooky issues I've seen in production involving multicast observables that are handed to third parties throwing and killing everyone else's subscriptions.

Sure, for those use cases (which are very similar to EventTarget), a swallowing observable would be appropriate. For those cases, you just do this. Swallowing should be a recipe, not a law.

[benlesh]

@zenparsing usually I recommend against imperative use of subject, though. I certainly wouldn't recommend handling errors in that manner. If there's any async down stream from that subject, errors there won't be caught. Really the errors should be caught in the handler in your subscription to the subject or whatever was composed off of it. It all seems pretty edge case.

[benlesh]

I guess I just don't see how you can compose asynchronous errors in two directions with try/catch. It seems weird that it would work sometimes, but not work all the time. Observable is unidirectional in part because of asynchronous push.

[zenparsing]

@blesh

We seem to be in a loop. It goes like this:

1. There are no use cases or practical uses for propagation (either return values or exceptions)
2. Wait, here's one
3. No, that use case is stupid
4. Start over

We're probably not going to make any headway like this. 😄

As I've stated before, in the server app that I'm working on at the moment, neither a global error handler (a la uncaughtException) nor a process crash are really acceptable behaviors for when an observer happens to throw an error in the context of a request.

and the workaround requires further explanation and comes with heavy performance penalties.

Please clarify?

[alex-wilmer]

try {
  subject.next(nextValue);
} catch (err) {
  // Do something reasonable
}

This seems highly unergonomic. Is this escape hatch worth a more principled approach to handling stream events?

Thinking about this more.

[zenparsing]

@alex-wilmer I'm not sure I understand what you mean? I was attempting to illustrate a general pattern that would be impossible to accommodate with swallowing observables.

[benlesh]

Please clarify?

Basically you have to schedule after every multicast with observeOn. It's not ideal.

No, that use case is stupid

I didn't say that. But I think you need to have a consistent API for error handling, and wrapping a subject.next() with try-catch will be inconsistent in behavior. It'll only work if everything you're doing downstream is synchronous. Disallowing it with error trapping will force consistent means of error handling. (Via subscribe, forEach, or a registry point)

On principle, I've never liked this behavior in promises, but I now understand it to be a necessity of the design.

[zenparsing]

Sigh...

[benlesh]

@zenparsing 😘❤️🤣

[jordalgo]

But I think you need to have a consistent API for error handling, and wrapping a subject.next() with try-catch will be inconsistent in behavior. It'll only work if everything you're doing downstream is synchronous. Disallowing it with error trapping will force consistent means of error handling. (Via subscribe, forEach, or a registry point)

@blesh - Just curious, is this an argument for the swallowing of errors on subscribe ? Also how does error trapping work if there is an async error in subscribe (as you were mentioning)? Wouldn't the error still get thrown regardless of if the Observable is swallowing sync errors? I'm a bit confused as to how that's more consistent (though it maybe a result of reading this thread 😉 ).

[benjamingr]

Please stop using the term swallowing errors. Promises do not swallow errors. I'm not sure how many times I'll have to reiterate it here.

@zenparsing's use cases are legitimate, and I'd like to discuss them - on the Node front, I suspect users won't be calling subscribe very much at all. They'll call forEach in their own code and .subscribe at a framework level.

For example, an observable-like express will call subscribe on a returned observable for you and all other subscriptions will be handled internally:

app.gets("/foo/:home", o => 
  o.filter(req => validate(req))
    .switchMap(req => fetchFromRedis(req.params.home))
    .map(processCache)
    .catch(e => fetchFromDatabase(e.path))
)

That way, you're not the one attaching .subscribe anyway and the consumer can be responsible enough to deal with resource cleanup and not throw - this is something the server-framework can solve for us in Node and I'm perfectly fine with putting the burden on the framework implementors and not the users in this case.

For this reason I think calling HostReportErrors in .subscribe given how .forEach works and the control needed for the multiple observers case is a necessary evil.

I'd love to hear whether this sort of ergonomics sounds like something you think users will enjoy.

[zenparsing]

A new day and new energy for debate. ☕️

Let me take a step back and summarize what seem to me to be the strongest arguments on both sides:

For current behavior (no communication boundary between producer and consumer)

• (A1) We should not introduce a primitive to the language that violates normal call stack exception propagation. Note that promises do not violate normal call stack propagation. Promise handlers execute in a fresh call stack per handler.*

For proposed behavior (one-way communication boundary between producer and consumer)

• (B1) Using an Observable subclass or wrapping the observer is an unnecessary burden for the only safe uses of Observable.

*By "normal" call stack exception propagation, I mean that if a function call results in an exception, then either the error is propagated to the caller using try/catch unwinding or the return value of the function is a rejected promise.

[zenparsing]

My counter arguments to B1:

• (B1-1) This is an opinionated statement coming from the context of a specific programming style (functional reactive programming). There are valid use cases for a push-notification primitive outside of that context.
• (B1-2) The burden is overstated. The best use case for one-way observables is multicast, when the multicast is distributed to disconnected, independent components. A multicast combinator can be written to use a HostReportErrors equivalent, and "hot multicast" observables can be written such that they wrap their calls to next. These abstractions sound like great candidates for a small NPM package.

[jhusain]

@zenparsing Simple question: do you expect every developer to wrap subscribe calls in a try/catch, and provide an error callback?

var observable = someFunc();
try {
  observable.subscribe({
    next(v) {
      // ...
    }
    error(e) {
      // async error
    }
  })
}
catch (e) {
  // sync error
}

It's not clear to me how developers avoid doing this without leaking the abstraction details of an Observable. Placing this burden on the developer doesn't seem to be defensible to me. In contrast, placing a try/catch in the Observer always works, sync or async.

var observable = someFunc();

observable.subscribe({
  next(v) {
    try {

    }
    catch (e) {
      // sync or async consumer handling error
    }
  }
  error(e) {
    // async producer error
  }
})

Sent from my iPhone

On Dec 20, 2016, at 7:49 AM, zenparsing notifications@github.com wrote:

My counter arguments to B1:

(B1-1) This is an opinionated statement coming from the context of a specific programming style (functional reactive programming). There are valid use cases for a push-notification primitive outside of that context. (B1-2) The burden is overstated. The best use case for one-way observables is multicast, when the multicast is distributed to disconnected, independent components. A multicast combinator can be written to use a HostReportErrors equivalent, and "hot multicast" observables can be written such that they wrap their calls to next. These abstractions sound like great candidates for a small NPM package. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[RangerMauve]

Just my two cents: As a user, I would find it most convenient if errors produced by my next handler were sent to my error handler, or potentially something downstream in the observable chain, so that I could perform cleanup if necessary.

[ljharb]

@zenparsing to clarify, is your contention that next should be able to throw, necessitating try/catch, or simply that an error thrown by a next handler be something you can respond to somehow? If the latter, do you need to be able to respond synchronously, or would async be ok?

[mattpodwysocki]

@zenparsing I'm going to have to side with @jhusain on this, that it's the job of the Observer to do the try/catch behavior, not the subscribe part of the Observable

[zenparsing]

@RangerMauve Unfortunately, I'm not sure we can do that now that errors from next calls do not perform cleanup. Currently next can now throw multiple times, but there's only supposed to be one terminating error call per stream.

[RangerMauve]

@zenparsing Fair enough.

@jhusain Your proposed code example seems reasonable to me. I would just worry about people forgetting to do a try-catch, and crashing the Node process like @zenparsing suggests. It is obviously "solveable" by having try catch blocks all over the place, but it feels more gross than having the container wrapping our code catching errors and sending them down a specific channel the way that Promises do.

As well, people might have to put the body of their code into another function to improve performance due to try-catch impeding optimization. Although that requirement seems to be changing.

[zenparsing]

@jhusain Currently, we try to send any errors which are thrown from the subscriber function to the observer's error method.

let observable = new Observable(sink => {
  sink.next(1);
});

// Logs "from observer"
observable.subscribe({
  next() { throw new Error('from observer'); },
  error(e) { console.log(e.message); },
});

Does that change the question?

[zenparsing]

@ljharb

to clarify, is your contention that next should be able to throw, necessitating try/catch, or simply that an error thrown by a next handler be something you can respond to somehow? If the latter, do you need to be able to respond synchronously, or would async be ok?

Without a doubt, there needs to be a non-global way to respond to errors. For observable, we don't have any hook like that. Do you have something in mind?

[ljharb]

One thought is that .next could return a promise - it could either never resolve, or resolve to the next value (whatever makes sense), but importantly, it would reject when next threw. You could respond to that error with normal promise mechanisms. All the use cases that didn't care about this could ignore the return value.

[benjamingr]

@zenparsing I'd love to hear your opinion on https://github.com/tc39/proposal-observable/issues/119#issuecomment-268234983 - namely:

• Users will use .forEach which works nicely with async/await, handles errors, returns a promise and so on. It's safe in the context of issues you're raising.
• Frameworks will use .subscribe which gives them a lot more control over handling errors and things they need. I'm perfectly fine with putting the burden on the framework implementors and not users to try/catch in this case.

[zenparsing]

Note: I updated my statement of the arguments to include a definition for "normal exception propagation".

@benjamingr I'm not sure whether forEach will work for all user-level use cases or not, since it tears down the subscription if the forEach handler throws. Perhaps users will still want to subscribe for event listener-type scenarios.

[benjamingr]

@zenparsing care to motivate a case where end users can't .forEach instead of subscribe or it's significantly harder to do so? I think while there might be such cases they're probably very rare .

[RangerMauve]

I've thought about it more, and I think that having users put try-catch blocks around places where they want to handle errors makes sense if you think of observables as EventTarget listeners. If I'm working with events on the client-side, thrown exceptions just get swallowed up and reported to host errors. This lines up well with the proposed changes.

On Node, I can't really guarantee that something thrown in an event listener will necessarily be handled anywhere. I could have code that's calling emit() somewhere in an async callback, in which case, unless I have to have try catch around every call to emit() if I don't want to have to do the try catch in the event listeners themselves. In this case, it's just a matter of whether the producer or the consumer will have to worry about exceptions. In a lot of the code I've seen, neither side worry about that and just let the process crash. I would argue that it wouldn't be too bad an idea to have it be the consumer that has to worry about that stuff in order to be consistent with both the browser JS ecosystem and Node JS ecosystem. Though I think that users should be warned that Observables are really not like promises at all when it comes to catching errors.

[zenparsing]

@benjamingr If I'm setting up an event listener that wants to keep listening to an infinite stream of events even if the "next" handler throws an exception, I'll probably reach for subscribe. That might be the case if I'm setting up an observer which is supposed to "stay alive" for a very long time (like the life of the application, perhaps).

[benlesh]

If I'm setting up an event listener that wants to keep listening to an infinite stream of events even if the "next" handler throws an exception, I'll probably reach for subscribe.

But the behavior is that an exception in the next handler will kill the observable stream.

// 2 and 3 will never be logged.
Observable.of(1, 2, 3)
  .subscribe({ next(x) { console.log(x); throw new Error('wat'); } });

Are you saying you want people to do this? (Really asking, because I'm confused)

try {
  Observable.of(1, 2, 3)
    .subscribe({ next(x) { console.log(x); throw new Error('wat'); } });
} catch (e) {
  // ignore
}

because I don't think that will work either. Even if the source was async, it would currently teardown the producer.