Closed tbondwilkinson closed 8 months ago
I do worry a little that people will expect the Observable constructor to behave similarly to the Promise constructor, in particular being executed synchronously and only once for every .then() callback.
That cat is sort of already out the the bag for more than a decade now. It's a non-issue in terms of broad understanding of the type. I know that's dismissive, so I'll try to elaborate.
However, to play devil's advocate, there are other APIs that don't quite line up with what the Promise
constructor does despite having a similar "look", for example MutationObserver
or ResizeObserver
:
const resizeObserver = new ResizeObserver((entries) => {
// do stuff
});
resizeObserver.observe(element);
resizeObserver.disconnect();
const mutationObserver = new MutationObserver((entries) => {
// do stuff
});
mutationObserver .observe(element);
resizeObserver.disconnect();
And honestly, those two APIs would have been so much better off as observables, because as they're currently implemented, while you can unobserve
different elements, if you observe
two different elements because you want to do something similar with each, you end up needing to add conditionals in the callback to handle them in whatever slightly different ways you might need to. As opposed to something like new MutationObservable(element).subscribe(callback)
or the like.
In terms of unicast/multicast, which is this issue:
const expensiveThingObservable = new Observable((subscriber) => {
subscriber.next(someReallyExpensiveComputation());
});
This is already something people face with ordinary functions:
const getExpensiveThing = (callback) => {
callback(someReallyExpensiveComputation())
}
And the solution is exactly the same:
let firstTry = true
let expensiveThing = null
const getExpensiveThing = (callback) => {
if (firstTry) {
firstTry = false;
expensiveThing = someReallyExpensiveComputation();
}
return expensiveThing;
}
Again, identical with an observable:
let firstTry = true
let expensiveThing = null
const expensiveThingObservable = new Observable((subscriber) => {
if (firstTry) {
firstTry = false;
expensiveThing = someReallyExpensiveComputation();
}
subscriber.next(expensiveThing);
subscriber.complete();
});
Now when it's asynchronous, it's just a little different:
let waiting = true
let expensiveThing = null
let deferred = new Set()
const expensiveThingObservable = new Observable((subscriber) => {
if (waiting) {
someReallyExpensiveAsyncThing((err, value) => {
waiting = false;
expensiveThing = value;
const subscribers = Array.from(deferred);
for (const s of subscribers) {
if (err) {
s.error(err);
} else {
s.next(value);
s.complete();
}
}
})
} else {
subscriber.next(expensiveThing);
subscriber.complete();
}
});
BUT: No one should be using an observable for an asynchronous single value, that's silly. Just use a promise for that. But even that comes with gotchas. For one thing, it's not an amazing idea to keep a promise around as a means of caching a value. Some promises can close over scope that is retained for the life of the promise, that's a runtime implementation thing, but it's still worth avoiding. So even with promises, the "expensive calculation" issue gets a little weird:
let promise;
let waiting = true;
let expensiveThing;
async function getExpensiveThingAsync() {
if (waiting) {
promise ??= new Promise((resolve, reject) => {
someReallyExpensiveAsyncThing((err, value) => {
waiting = false;
if (err) {
reject(err);
return;
}
expensiveThing = value;
resolve(value);
})
});
return promise;
}
return expensiveValue;
}
In the end, Observable
is a very low-level type, Promise
is a low-level type. Neither are engineered to solve every problem, rather they are building blocks to allow such things to be built. Observable
is actually a little more "low-level" than Promise
in a number of ways, and could be used to implement promises (or a promise-like thing) but not the other way around.
But are there API decisions that could alleviate this concern?
Like an Observable constructor could look more like ReadableStream:
const subscribers = new Map();
const observable = new Observable({
subscribe(subscriber) {
subscriber.next(1);
...
subscribers.add(subscriber, metadata);
},
abort(subscriber) {
...
subscribers.delete(metadata);
},
});
In this example, the Observable
constructor doesn't look much like Promises at all, and so it's unlikely to cause confusion. It also somewhat alleviates the concerns of #3, because now the constructor does have a teardown method.
This approach doesn't change behavior at all, it just merely changes the shape of the constructor so that it's less of a Promise-look-alike.
Like an Observable constructor could look more like ReadableStream
That would really complicate the type.
You can do what you're proposing above pretty easily with the proposed API:
const subscribers = new Map();
const observable = new Observable((subscriber) => {
subscriber.next(1);
...
subscribers.add(subscriber, metadata);
return () => {
subscribers.delete(metadata);
};
});
the Observable constructor doesn't look much like Promises at all, and so it's unlikely to cause confusion.
I'm not sure how to phrase this without it reading as snarky, so apologies in advance: But new Observable(() => {})
and new Promise(() => {})
already look different because they're different constructors. (same with new MutationObserver(() => {})
and new ResizeObserver(() => {})
et al). Observable
and Promise
have been in such broad use for the last decade that I don't think there's a big risk there. I mean, one could argue that function* () { return 'hi'; }
vs function() { return 'hi'; }
is problematic because they return different things when you call them, and the only difference there is just one character... Or async function() {}
vs function() {}
... I don't think new Promise()
vs new Observable()
is likely to be a footgun.
Fair enough, thanks for your responses and thoughts.
Probably worth leaving open in case someone else has the same thought, but as far as resolution it's your speculation that people won't get confused vs mine that that they will, and your speculation is based on a lot more experience with people using Observable
, so I tend to weigh yours a little higher.
But it could be worth finding a more rigorous way to answer this question by reaching out to a broader set of people to hear their opinions as well. Now is the time to figure out if API shape deviations from RxJS Observable
is worth doing.
Tom asked me to chime in, so my thoughts:
I don't share the concern that people will confuse the Observable constructor's callback with the Promise constructor's callback. As Ben points out, there are other callback-accepting constructors on the platform, with different behaviors.
On the other hand, I think Ben's arguments analogizing to ordinary functions remain confusing and unlikely to help people understand. (Similar to #6.)
I think the ReadableStream
-like-constructor idea is very intriguing. I think that should primarily be decided by its relation to #3, or perhaps more generally, based on whether there are other things besides the subscribeCallback
function which need to go into creating an observable. For a ReadableStream, there are very clearly several ingredients needed: start
, pull
, and cancel
callbacks, and then some configuration options: type
and autoAllocateChunkSize
. Could that be the case here?
I think the idea of returning your cancellation callback is very un-idiomatic for the web platform. An options bag version would be much more normal, from that perspective.
I don't find arguments around "complicating the type" compelling, or arguments that people should roll their own solution to a common problem.
But, maybe there are other solutions to #3, or maybe there will never be a need to configure an Observable
with more options than just the subscribeCallback
. In which case the current design fine. I don't see a need to move to the options bag style just to differentiate from Promise
.
I think Ben's arguments analogizing to ordinary functions remain confusing
That's fair, it's not for everyone.
I think the ReadableStream-like-constructor idea is very intriguing
It is, but I'm concerned about introducing anything that isn't battle tested. One thing that the original proposal for Observable for the TC39 lacked was a way to set up teardown during the "start" of the subscription. RxJS solved this by having an add(teardown)
method on the Subscriber, and we have a signal on the Subscriber here to handle some of that.
That said, I'm open to an API like this if it works and makes sense, as it could easily be added to existing implementations like RxJS in a non-breaking way.
Here's a use case, just as an example, perhaps someone sets up an Observable that starts two different resources, that could error during start up:
There's a lot for the user to do to make sure this is safe, which is mostly okay, because it's a pretty basic type. Yow might have to do similar things to reject in a Promise
constructor.. The twist is you want to make sure teardown happens. So a try/finally must be added to make sure the teardown function is returned in the event of an error.
To complicate things, the returned teardown function is actually added to the subscription after that subscription would already be unsubscribed (implicitly by the subscriber.error
call). So you have to make sure that a closed
subscription will automatically execute any new teardowns someone tries to add to it.
new Observable(subscriber => {
let resource1;
let resource2;
try {
resource1 = new Resource1(value => {
subscriber.next({ from: 1, value });
});
resource2 = new Resource2(value => {
subscriber.next({ from: 1, value });
});
} catch (error) {
// make sure to communicate the error to the consumer.
subscriber.error(error);
} finally {
// make sure we attempt to teardown
return () => {
resource1?.disconnect();
resource2?.disconnect();
}
}
})
RxJS attempts to handle all of this by automatically forwarding any errors thrown in the "start" to the subscriber.error
, and it also provides a way to add
teardowns to a Subscriber
(which inherits from Subscription
).
This is a little more straightforward, and could easily be rewritten to use subscriber.signal.addEventListener('abort', () => {})
instead of subscriber.add(() => {})
(but you can see the ergonomics issues there.). It's also worth noting that the teardown has been added to the subscriber/subscription prior to the error being thrown and forwarded, so there's no fancy logic really necessary when it comes to dealing with teardowns after the subscription has already been closed by an error.
new Observable(subscriber => {
let resource1;
let resource2;
subscriber.add(() => {
resource1?.disconnect();
resource2?.disconnect();
});
resource1 = new Resource1(value => {
subscriber.next({ from: 1, value });
});
resource2 = new Resource2(value => {
subscriber.next({ from: 1, value });
});
})
This has a few issues. For one thing, how do we create a new resource per-subscription, to share between the two callbacks? We'd need to have some sort of state setup function or the like?
new Observable({
start(subscriber) {
// Todo, create two resources.
},
cancel() {
// Todo, teardown those resources if necessary
}
})
We could do something to have a callback that initializes some shared state on subscription, but it seems needlessly complicated.
new Observable({
initState() {
return { resource1: null, resource2: null }
},
start(subscriber, state) {
state.resource1 = new Resource1(value => {
subscriber.next(value);
});
state.resource2 = new Resource2(value => {
subscriber.next(value);
});
},
cancel(state) {
state.resource1?.disconnect();
state.resource2?.disconnect();
}
})
Alternatively, we could have something that returns the configuration from within a closure to allow the resources to be created, but I have zero doubt that people will mess this up as well
new Observable(() => {
let resource1;
let resource2;
return {
start(subscriber) {
resource1 = new Resource1(value => {
subscriber.next(value);
});
resource2 = new Resource2(value => {
subscriber.next(value);
});
},
cancel() {
resource1?.disconnect();
resource2?.disconnect();
}
}
})
Anything else I'm thinking of here seems like it would force eagerness and multicast (closing over outside instances of the resources, manual reference counting, etc), which really dilutes the possible uses of the type and really makes it not an observable.
The crux of it is that the start
function is really responsible for creating the things that you'd want to cancel
. Therefore it makes sense to hand control over registering cancellation/teardown to the start
function, which brings us back to the original (very battle-tested) design.
(I'm truly sorry that all of this ends up being a novel, I'm just trying to be thorough 😅 )
The way we'd handle that with the streams-like pattern is as follows:
new Observable({
start(subscriber, state) {
this.resource1 = new Resource1(value => {
subscriber.next(value);
});
this.resource2 = new Resource2(value => {
subscriber.next(value);
});
},
cancel(state) {
this.resource1?.disconnect();
this.resource2?.disconnect();
}
});
I'm amenable to this design. My only concern would be it's new. I'd want to fiddle with it to see if it was able to meet the same needs/use cases. One concern is that it would force it to be multicast, or require ref counting. But I suppose it could be made to not work like that. But maybe that's not a big deal. It requires thought.
I guess the addTeardown()
method that was recently added is an alternative to the API shape above, providing the teardown to the constructor? I still think the constructor version is more idiomatic...
So we're discussing the possibility of having a bag of options passed into the Observable constructor, instead of just a callback which is responsible for managing teardown from within itself. This means we'd have to have a way to share state between the passed-in start/subscribe callback, and the stop/cancel/teardown callback. But I'm a little confused about what was posed in https://github.com/WICG/observable/issues/22#issuecomment-1586711085. There are three pieces of state there:
this
: I actually can't tell if this
points to the Observable instance (shared between all subscriptions) or the object passed into the constructor that holds the two callbacks (also shared between all subscriptions). I think it's the latter, but either way it seems like it's not a per-subscription based object, right?subscriber
: This is clearly the per-subscription based objectstate
: I'm not sure what this is, in part because it is never used. Is it also a per-subscription based piece of state that's just loosely connected with subscriber
, so we don't have to tack on random properties onto the Subscriber
object? I can't tell what this would represent.
this
: I actually can't tell ifthis
points to the Observable instance (shared between all subscriptions) or the object passed into the constructor that holds the two callbacks (also shared between all subscriptions). I think it's the latter, but either way it seems like it's not a per-subscription based object, right?
The latter was the intent.
But if I understand correctly, this is bad: you need per-subscription state, and there's only a single per-Observable
object being passed in my example.
If I understand correctly that you need per-subscription state, then I rescind my idea: it is not a good fit and will be a footgun.
Given that I'm not sure where we are with a solution to this issue. My idea, with this
, had the benefit of being (IMO) a natural JS pattern, and one that the web platform already uses with Streams. Without that I think we need to go back to considering less-nice patterns, whether it's the original cleanup-function-returning, or something like encouraging people to tack pieces of state onto the subscriber
object, or something else.
Without that I think we need to go back to considering less-nice patterns, whether it's the original cleanup-function-returning, or something like encouraging people to tack pieces of state onto the subscriber object, or something else.
I definitely share your opinion about not liking the "return a cleanup function" from the subscriber callback (i.e., the callback that gets passed into the Observable ctor, just so we're all clear). It seems very un-idiomatic. Since the Subscriber
object is the obvious per-subscription mechanism, then I think per-subscription cleanups would have to be associated with it. So I subscriber.addTeardown()
makes a lot of sense there, and naturally lets each invocation of the subscriber callback register its own cleanup(s) etc.
Currently the explainer & Chromium implementation (behind a flag) has been using the addTeardown()
design expressed here, to add per-Subscription teardown callbacks. I think this design is satisfactory so I'm going to close this issue, but happy to re-open if we want to continue the discussion.
To preface, I understand that RxJS and other Observable implementations are widely used and very popular, and that a lot of people have been very successfully using Observables for very long time in many applications.
But there's a difference between using a library and using a web platform primitive. And I do worry a little that people will expect the
Observable
constructor to behave similarly to thePromise
constructor, in particular being executed synchronously and only once for every.then()
callback. Observables don't really work that way, as the callback parameter is called whensubscribe
is called, an unlimited number of times. This is the much discussed cold vs hot distinction.So for example, maybe it's really easy to write:
And people comfortable with promises will think that's only going to be executed once and then each subscriber gets the expensive value, when in reality the expensive computation happens once for every subscriber.
This is less of a concern for Observables that the platform creates for something like event listeners, where the browser is in charge of the observable instantiation and the rest of the API is just really offering a better listener pattern.