Where does the language end, and the framework begin?

[jrista]

As an avid fan of Rx in general, and RxJS specifically...I feel the question has to be voiced: Why do we need Observable in the language? Bringing Promises in made some sense, given their ubiquity and the way a true language feature, async/await, was able to build upon them.

However with Observables, the proposal here seems like the beginnings of sucking in an entire, rather large scale framework like RxJS. Especially once discussion begins on operators and Subjects and the like.

So where does the language end...and the framework begin now? Are we trying to do too much inside of the language of ECMAScript? If the Observables proposal remains as simple as it is now, then what is it really buying us, if a large library like RxJS (or a library of operators) is still necessary to get the fully featured and rich functionality we are already accustomed to in our daily work? We would not really save much or anything in terms of bundle size for web apps.

How would the introduction of Observable into the frameuage (or langwork, if you prefer) of ECMAScript impact continued future development of RxJS? Rx itself won't be going anywhere, as it is a well established, cross platform, cross language ecosystem of frameworks that has extensive support from a wide variety of communities. Will RxJS suddenly find itself in a bind, with its own classes conflicting with those of the frameuage implementations? Would a frameuage Observable be extendable, allowing the addition of operators over time, or allowing for a flexible API to hook operators in, supporting richer functionality? Would operators be a part of the frameuage implementations, or would that explicitly be left out (thus leaving RxJS fans to wonder again...why?)

Why do we need ES Observables, and is it really appropriate to include in the language? Is it really just a framework, and should it remain a framework?

[benlesh]

It's a primitive that's actually much more primitive than even Promise. It's as primitive as a function, or a Map or a Set.

It supports:

1. synchronous push
2. asynchronous push
3. multiple values
4. cancellation
5. deterministic memory management (via teardown mechanism)

... and frankly there are no other primitives that provide these things in JavaScript.

Also, it can be treated like a linear collection of values, which means you have all sorts of operations you can perform on it.

You can actually build a Promise from Observable, you cannot do so the other way around. If Promise, Map and Set belong, Observable belongs, IMO.

[staltz]

... and frankly there are no other primitives that provide these things in JavaScript.

Actually there are: functions!

I actually begin to agree with the OP, there is no point in putting this into the language unless there will be new syntax. Promises into the language made little sense, but now with async/await as syntax sugar, Promises make sense. Same argument for Iterables. But Observables? We already have functions, the ultimate primitive, in the language. Observables are built on top of them.

[jrista]

Hmm, how are we defining "primitive" here...? I don't know that I would call Set or Map an actual "primitive" (not, at least, as I understand the term), and "Observable" has complex functionality that requires many language constructs (as clearly laid out in this proposal), which also seems to fall well outside of the bounds of a "primitive" language feature.

[benjamingr]

The idea is to add observables to the language so that host APIs like the DOM can use them for EventTargets and EventEmitter events (in Node).

If the language had an Observable type language APIs can start returning observables instead of addEventListeners which can be concisely consumed by libraries like RxJS or other host APIs.

The same way fetch and import return a promise today (since it's part of the language), event APIs like a document.body.clicks observable could exist natively (websocket event and webrtc messages would also rock).

Adding Observable to the language lets the platform use it in its own APIs which would save a whole layer of wrapping code and enable a world of tooling possibilities (since it's part of the platform).

[staltz]

language APIs can start returning observables instead of addEventListeners which can be concisely consumed by libraries like RxJS or other host APIs.

Language APIs could also just return subscribe functions instead of Observable objects.

[jrista]

Ben, thank you for your reply. I understand the idea there. I also like what Andre has stated, that functions are really the ultimate primitive, however sometimes primitive functionality is not quite capable of serving the need. You've clarified the need here...websocket events, webrtc messages, etc.

So the goal, then, if I understand, is NOT to try and replicate the full bodied richness of a framework like RsJS (I wondered, because this proposal does mention that operators could be in a future proposal, which is actually why I posted this)? The goal would be to keep the Observables within the language simpler, allowing a framework like RxJS to consume them and extend them with it's extensive richness? Is extensibility being considered, and will it be first class? It still wonder about the naming conflicts between a language native Observable and an RxJS observable, and whether that would be a problem.

[js-choi]

Regarding the general question, there was a relevant “Open-ended discussion: How should we evolve the JavaScript standard library over time?” scheduled in the agenda for the 62nd TC39 meeting several weeks ago. @littledan and @bmeck had created a slide presentation to start the discussion. Unfortunately, it seems that they were not able to get to that topic during the meeting, according to the meeting notes. It is a general question that TC39 will consider, however.

[benlesh]

Actually there are: functions!

Now we're getting silly. "We don't need functions if we just add goto!"

Observables are a primitive that wraps a lot of really interesting and important features in a lightweight package that are non-trivial for the majority of people to just roll out of functions. Granted, there are libraries that do this for them, but native APIs can't pull in third party libraries for these, so they'll have to use their own.

If native APIs (like the DOM, or Node) start using their own flavors of Observable, then we have a problem: Different APIs everyone has to remember that all do the same thing. Basically this is the same issue we see with addEventListener vs addListener vs on. We were starting to see that with Promises until they standardized, but people don't see to recall that. Different promises had different behaviors, different ways of creating them, different methods outside of then... It wasn't good.

This is about keeping the JavaScript eco system sane. Or at least... more sane that it might be otherwise.

[staltz]

Yeah Ben, but not really. We will still have callback-only APIs for async, still have Promise-only APIs for async, and Observables would be just one more primitive. Different ways of interfacing that everyone has to remember, which could have been designed with the same interface.

And when I talked about just functions, I'm also quoting you. You're the one who wrote about "observables are just functions that take an observer and return a subscription".

We could have platform APIs just return a function subscribe(observer) => subscription and still "wrap a lot of interesting and important features" and still be able to write pipeable operators for those functions.

What I'm questioning here is what's the necessity for an object wrapping the subscribe function.

[benlesh]

Providing all of the guarantees that Observable provides is non-trivial, we both know that. When I say "observables are just functions" I'm trying to simplify what they are to promote understanding.

Standardizing the API in JavaScript will save us from a fate where there are competing observable implementations in native APIs and popular frameworks, just like it did for promises. It will pave the way to powerful new syntax, just like it did for promise.

I think this issue, while it raises some solid points, is missing the forest for the trees.

[jrista]

Ben, can I ask...how would this work with RxJS (or similar libraries) in the future? Would we be limited to the basic specification described in this proposal in the future with observables (losing the rich operator libraries (which I must thank you profusely for! You are a prolific developer! :))?

I am not sure that this proposal addresses any way to add operators. It also does not seem to cover a pipe() function. It notes that find and map may find their way into a future spec...but who knows when, and find and map are a tiny fraction of the rich functionality we have now with RxJS.

I think as long as this proposal can work with libraries that extend it, then it is not an issue. If it somehow conflicts with existing libraries, forcing quirky workarounds, then I see that as an issue that should be resolved now, rather than later.

I will say this, I think the comment by js-choi termed it right: The JavaScript (or ECMAScript) Standard Library. I don't think I would call Observable a primitive, it would be part of the standard library. That is a "framework," and perhaps it does belong there. The standardization of Promise was a good thing...but along with it's folding into the standard library came direct language enhancements as well.

I recently came across the pipeline operator, which is like a forward apply/forward pipe from functional languages, which seems like it could make functional programming patterns with Observable a lot easier to deal with. A forward apply would be a true language feature, and a true primitive, that could build on top of Observables like async/await built on top of Promises. I think that would be very, very interesting.

[jaawerth]

A few thoughts on why I'm a fan of this proposal. Sorry for the wall of text - reading all these comments got me thinking enough to rant a little..

Symmetry

Observables are to Promises as Arrays are to Values

Simpler than EventEmitter!

Node's EventEmitters are nice and clean, but opinionated presence of named events make it harder to create higher-order methods for them without having to juggle how they work propagate. Observables are actually simpler - they're just subscribable streams of values onto which a named event interface can optionally be grafted, and can act as composable glue between browser events, EventEmitter, Node streams, and anything else that represents a time-agnostic sequence of values.

The "start," "next", "complete," and "error," event types are optional and by existing as function hooks they prevent the need for event namespacing for these fundamental conditions.

The simplicity also makes any higher-order methods like map, filter, flatMap, etc easy to use and reason about, and others easy to implement in userland.

Interop

What I'm questioning here is what's the necessity for an object wrapping the subscribe function. - @staltz

I think having both an object wrapper with Symbol.observable protocol is preferable so a stream lib implementer can add a universal, first-class compat layer analogous to Iterables/Iterators (which also allows for native .from and syntax tie-in ala for..on). Using a constructor/class rather than a subscribe function makes it a universal API, and making it an object means you can pass it into other functions for a two-way API.

In userland, we've got RxJS, kefirjs, baconjs, most, highland, js-csp (sorta), and more all for dealing with reactive streams of values in addition to what we have in node in different ways. This is a sign that this is an important part of the flow control toolkit, and until recently they all suffered from interface fragmentation, not unlike Promises did before A+ and ES spec ($q.defer, etc).

This proposal's very existence has already improved the state of things: RxJS has already made it a stated goal to track this proposal for its core Observable object, and both kefir and most have added bidirectional methods that leverage Symbol.observable.

As a user, I can write functions that manipulate observables without caring which of the userland implementations they come from, and rely on the protocol and spec to connect the dots.

Higher order methods

@jrista - you make a good point about higher-order methods. I think it's good to have a simple native object and leave most of the utilities to userland, but simple ones like map, filter, etc. I would also like to see a .then(onComplete, onError) method; making them thenable allows for Promise interop via Promise.resolve and await.

.pipe would be nice to have, but wouldn't make sense for an outer API because you shouldn't be able to pipe into an observable returns from an external data source that implements Symbol.observable - an Observable is more like a Readable stream in that sense. I imagine that's why Rx keeps that behavior separate in Subject. Also, this can be covered natively via Observable.from(otherObservable), which has similar behavior to pipe.

[benjamingr]

@staltz we discussed the idea extensively in the past - it's really interesting discussion IMO https://github.com/tc39/proposal-observable/issues/34 , and https://github.com/tc39/proposal-observable/issues/11 is also worth a read

[benjamingr]

@jrista

I am not sure that this proposal addresses any way to add operators. It also does not seem to cover a pipe() function. It notes that find and map may find their way into a future spec...but who knows when, and find and map are a tiny fraction of the rich functionality we have now with RxJS.

The idea is to create a language level interface now and to add the operators later for spec simplicity.

I think as long as this proposal can work with libraries that extend it, then it is not an issue.

That is explicitly the case.

@jaawerth

Observables are to Promises as Arrays are to Values

That's not true, Observables/Async Iterators/Streams are to Promises/Tasks as iterables are to values might be a little better. I recommend @headinthebox's https://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2014/Keynote-Duality as a must-watch. Also check out https://github.com/kriskowal/gtor .

[jhusain]

The rationale for Observable’s inclusion into the language continues to be its usefulness to the web platform. The champions are following the path paved by the Promise proposal (ie add primitive for use in platform, add combinators over time). I think that concerns about this proposal potentially pulling a large set of combinators into the language are overstated. The champions would have to demonstrate to the committee that these methods were sufficiently universal as to warrant inclusion in the language (a high bar). As an example, Array has not absorbed all of the methods in lodash.

There will always be some methods that live in userland libraries, but Symbol.observable allows for easy interop.

It is of course true that Observables can be represented as functions, but so can Promises and many other types (ie “lambda the ultimate”). Under the circumstances, modeling Observables as functions but not Promises would seem to violate the principle of least surprise.

[jaawerth]

@benjamingr

That's not true, Observables/Async Iterators/Streams are to Promises/Tasks as arrays are to iterables might be a little better. I recommend @headinthebox's https://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2014/Keynote-Duality as a must-watch. Also check out https://github.com/kriskowal/gtor .

That's fair - I've read quite a bit of gtor, but admittedly lost some truth to the point I was making in the attempt to boil it down to a single sentence. I'll check out that video, though - thanks!