Alternatives to xstream?

[megri]

XStream seems to be the one thing that forces a build to use the ScalaJSBundlerPlugin (this caught me when initially trying the library).

Is there a specific reason that XStream was chosen over something like Monix?

[raquo]

Yeah, I get asked this a lot since XStream.js is not native to Scala. My two primary criteria for Laminar's streaming solution are:

1) Conceptually simple. XStream streams are synchronous, "hot" / multicast (executed only once per event, not once per event per listener). There are no schedulers, execution contexts, backpressure strategies / etc. It is very beginner-friendly.

2) Small download size. XStream is less than 40KB in production. For comparison, as far as I can tell Monix is around 200KB, which is significantly more. Not just because of the download latency, but because all that code needs to be unzipped, parsed and loaded into memory. Especially on mobile devices that's a big problem. Monix is modular, so it might be possible to reduce download size by only requiring the monix-reactive package, but I didn't have the time to try that.

It helps that XStream was built specifically for Cycle.js, a use case that is very similar to Laminar. Here its author who also has significant experience with RxJS explains why he decided to go that way:

https://staltz.com/why-we-built-xstream.html https://staltz.com/why-we-actually-built-xstream.html https://staltz.com/cold-and-hot-callbacks.html

---

Because XStream is so simple and lightweight, if we ever come up with a way to use different streaming solutions in Laminar (relevant discussion), it will probably be a good lowest common denominator due to lower overhead.

---

One more thing – in the next version of Laminar I will try to make it impossible to forget to unsubscribe from a stream. So the stream.addListener(listener) method will gain another implicit parameter, something like subscriptionLifecycleContext which could for example be a Laminar element. Then the subscription will be cancelled when said element gets unmounted.

With XStream, I can do this reliably because I can modify the XStream.scala interface to suit my needs. In Monix, from my understanding the only thing we could do is add an optional syntax for the same (example) which still leaves the original .subscribe/.addListener method exposed for abuse.

---

On the other hand, what I don't like about XStream:

1) It looks weird to Scala people, that could hurt adoption of Laminar 2) Writing a typed interface to it was annoying. I would prefer a native Scala solution

---

I'm not 100% locked to XStream, but it seems like the most compelling solution to me at the moment. I am especially on the lookout for a single solution that will have both streaming and state management functionality like Felix described here: https://github.com/OutWatch/outwatch/issues/96#issuecomment-346654224

[raquo]

btw is there an issue with requiring ScalaJSBundlerPlugin? I was under the impression that any real world Scala.js project will end up using it to get some JS dependency or other... For building SPAs I figured that would be mostly inevitable.

[megri]

It feels like the plugin alters the build in a non-transparent way (requirement of installing yarn, changing artifact paths etc). Maybe it's just me; I'm still quite unfamiliar with Scala/JS but is hoping to use it for a mobile first web-app soon. It's not a deal breaker but it'd feel nicer if the toolchain was 100% Scala.

Thanks for the links, very interesting read! So streams being hot is desired in order to avoid the retriggering of side-effects when connecting observers to some stream that is side effecting. Is that a correct interpretation? Does that relate somehow to push- vs pull-basedness of streams? Monix should be push by default if I'm not mistaken.

Not having to deal with an execution context is nice, and not having to unsubscribe listeners is nice too! It does feels a bit strange to add this to the façade layer since it effectively alters the perceivable behaviour of the underlying library, but at the same time it feels practical.

Another approach would be fully abstract away the underlying stream library and require only those abstractions in the binding API. This decoupling would also make it possible to try different libraries without touching the surface area. Maybe something like https://github.com/lihaoyi/scala.rx would work. Seeing as its only dependency is the Scala library I (perhaps naïvely) imagine it would be a very lightweight solution in terms of added kB.

[raquo]

So streams being hot is desired in order to avoid the retriggering of side-effects when connecting observers to some stream that is side effecting. Is that a correct interpretation?

Not exactly. Each listener would still be triggered once. A stream being hot means that non-side-effecting operators are also executed only once per event. So if you have myStream = otherStream.map(foo), and otherStream emits an event: if myStream is a cold stream, foo will be executed as many times as myStream has listeners, whereas if myStream was hot it would execute only once, and only if myStream has any listeners.

The benefit of hot streams is that it's more forgiving. If you have a side effect (such as local state mutation) in your .map somewhere it will still work like you'd expect, even though your expectation is technically incorrect for the FRP paradigm. FS2 library for example is on the opposite side of the spectrum. I haven't looked too deeply, but I think instead of using hot streams it lets you achieve safety by encoding side effects of your streams in types... or something like that.

Pretty sure XStream is also a push based system. IIRC a pull based system is where your observers actively query their dependencies for current state instead of listening to incoming changes, a very different system.

---

It does feels a bit strange to add this to the façade layer since it effectively alters the perceivable behaviour of the underlying library, but at the same time it feels practical.

This is exactly how I feel as well. I like to think that perhaps xstream.js might have had this feature itself, if Javascript had implicits.

I am also considering a separate wrapper layer like you said, but realistically it will not be able to offer interop between underlying libraries as those are too different once you get past the simple .map/.filter mechanics.

---

Regarding Scala.rx specifically, I looked into it again yesterday, and realized that it uses Scala's WeakReference to prevent memory leaks. Meanwhile, I think Scala.js does not really support weak references – the WeakReference class is stubbed to provide a strong reference due to lack of native WeakReference support in JS, as far as I can tell. So it's possible that Scala.rx requires even more careful memory management in Scala.js than its docs suggest (actually, cc @fdietze – this could be relevant to your Duality fork, or maybe you know something I don't).

Another issue is that Scala.rx does not support empty vars / observables, so you can't really do streams with it, you can only represent time-varying values aka state. While it is definitely possible to modify Laminar to work like that, typical user code would look quite a bit different. Not a deal breaker, but something to consider carefully.

---

After looking at Scala.rx and reactive-core, I think Laminar would work best if we had a library that covered both state management and event propagation, but would also have more manual memory management (meaning that each listener/observer needs an implicit LifecycleContext, which Laminar would provide). As opposed to relying on WeakReference-s. That could potentially solve both FRP glitches and automatic memory management.

However, as far as I can tell, there is unfortunately no such library. I am building Laminar in order to build things with it, so as much as I'd love to implement such a library myself, I don't think I'll have the time anytime soon. It might be possible to fork either scala.rx or reactive-core to achieve this, but that's still a lot of time.

[fdietze]

I'm not 100% locked to XStream, but it seems like the most compelling solution to me at the moment. I am especially on the lookout for a single solution that will have both streaming and state management functionality like Felix described here: https://github.com/OutWatch/outwatch/issues/96#issuecomment-346654224

Sadly, I doubt that there will exist a library which does event-propagation and state-management well any time soon. Thats why we're trying to implement type-classes for both (https://github.com/fdietze/outwatch/blob/typeclass/nomad/src/main/scala/nomad/core.scala) and let the user decide if they want to use either

1. Two separate libraries (two apis + more payload)
2. Only Event-propagation / streaming like rxjs, xstream, monix (frp glitches if used for state-management / rendering UI https://github.com/monix/monix/issues/467, https://github.com/OutWatch/outwatch/issues/96#issuecomment-346957347. Always needs to be started with a default value, else UI is in an undefined state: https://github.com/OutWatch/outwatch/issues/137)
3. Only State-Managemnt like scala.rx (few operators, dataflow-graph-overhead - which is probably neglible in web-applications). To be fair, scala.rx was not only developed for state-management and brings some operators for handling events. It's pretty usable for event-management but lacks lots of operators that rxjs/monix provide. I'm using it in my projects for over a year now. I think these kinds of libraries are the most promising ones, if you one wants to use only one library.

Regarding Scala.rx specifically, I looked into it again yesterday, and realized that it uses Scala's WeakReference to prevent memory leaks. Meanwhile, I think Scala.js does not really support weak references – the WeakReference class is stubbed to provide a strong reference due to lack of native WeakReference support in JS, as far as I can tell. So it's possible that Scala.rx requires even more careful memory management in Scala.js than its docs suggest (actually, cc @fdietze – this could be relevant to your Duality fork, or maybe you know something I don't).

I'm not aware of anything like this. Could you please point me to the code?

One more thing – in the next version of Laminar I will try to make it impossible to forget to unsubscribe from a stream. So the stream.addListener(listener) method will gain another implicit parameter, something like subscriptionLifecycleContext which could for example be a Laminar element. Then the subscription will be cancelled when said element gets unmounted.

Just to mention, the same is implemented in outwatch (https://github.com/OutWatch/outwatch/blob/4a2f0469a043d8ea65ffd2a7b125e16d00f5c42f/src/main/scala/outwatch/dom/helpers/Snabbdom.scala#L111), but this covers only one part of the possible leaks (ideas: https://github.com/OutWatch/outwatch/issues/121).

However, as far as I can tell, there is unfortunately no such library. I am building Laminar in order to build things with it, so as much as I'd love to implement such a library myself, I don't think I'll have the time anytime soon. It might be possible to fork either scala.rx or reactive-core to achieve this, but that's still a lot of time.

scala.rx has a small and simple code-base and was easy to fork for me (https://github.com/fdietze/duality). I'm very open to discuss the direction of this project, maybe we can achieve something useful together. 😉

[raquo]

Just to mention, the same is implemented in outwatch

Yep, to that extent it is also already implemented in Laminar, even managed subscriptions (ReactiveElement.subscribe), but the problem is that managed subscriptions are opt-in, the user can still trivially bypass it because the underlying streaming library does not require a lifecycle context for creating subscriptions. I want to make it impossible to create a subscription without at the same time specifying when it should be destroyed.

---

Regarding weakrefs:

https://github.com/scala-js/scala-js/blob/master/javalanglib/src/main/scala/java/lang/ref/WeakReference.scala

https://github.com/scala-js/scala-js/pull/170 <-- here the description essentially says that weak refs are not weak.

I think WeakReference can potentially be implemented using Javascript's WeakMap such that you would be putting weakrefs into keys.

---

I am still trying to figure out how Scala.rx handles propagations to avoid glitches or excessive recomputing.

I think it should be possible to define a very simple propagation method where the changes in a Var propagate to child Rx-s until they meet an Rx that depends on multiple parents. Then when there is nothing left to do the propagation "continues" by computing those multi-parent Rx-s and continues on to their children using the same logic.

Lastly, when the change was propagated through the whole graph, only then we fire any triggers on the graph, in the order in which the propagation encountered them.

I think this could be a very predictable propagation technique assuming that it actually works on complicated graphs (even including partially recursive ones, I guess). I do expect unexpected behaviour in certain edge cases e.g. if you specify a trigger that updates the same graph that you've just finished propagating. Not sure if those issues would be solvable. Maybe such weirdly recursive code should just not exist.

[fdietze]

Here is the implementation of the Algorithm in scala.rx: https://github.com/lihaoyi/scala.rx/blob/master/scalarx/shared/src/main/scala/rx/Core.scala#L119

[fdietze]

Regarding WeakRefs in scala.rx: I digged a bit deeper and found this: https://github.com/lihaoyi/scala.rx/issues/21#issuecomment-174139432

[raquo]

Hrm. Well, then it means you need to remember to kill observers manually... I guess?

I did read Scala.rx propagation algorithm but its plain-English approximation is eluding me. It seems to be some kind of breadth-first search that doesn't really help solve redundant updates of Rx-s in principle, it only removes redundancies when firing triggers. I could be wrong, it's hard for me to read.

---

Despite my best judgement I went and tried my hand at making a very simple state propagation lib: https://github.com/raquo/laminar/pull/8 As if I didn't dig enough rabbit holes for myself... I guess I'll be adding more tests to it to see how it handles edge cases, and if that works out I might implement memory management for it and integrate it into Laminar.

[fdietze]

Hrm. Well, then it means you need to remember to kill observers manually... I guess?

Can you please try to produce such a leak in scala.rx? I'm interested. As far as I understood, this should not be possible.

I'll take a look into the scala.rx propagation algorithm soon.

What was your motivation for a new project instead of building on scala.rx? Understanding the principles? Because it could be possible that you end up building a clone. In my opinion scala.rx got a lot of design decisions right. The readme is worth a read. You can probably reuse some tests.

The motivation for my fork were bidirectional state operators and to drastically simplify the code base (also for understanding what's happening).

[raquo]

Can you please try to produce such a leak in scala.rx? I'm interested. As far as I understood, this should not be possible.

Sorry, I only have time to armchair this :(, stretching myself very thin as it is. Having looked at Scala.rx code, I see that its Dynamic Rx-s keep track of both upstream, downstream, and observers. On the surface it seems like nothing in that graph will be garbage collected unless each and every of those Scala.rx objects has no outside references to it (which might as well be never).

But, I could be wrong. It's hard to understand what exactly Scala.rx does with its upstream/downstream properties. It... seems to clear and re-calculate them on every propagation or something like that. Maybe that dark magic is what effectively substitutes for weak refs, but I honestly can't tell at this point. It doesn't seem like it should be possible.

---

What was your motivation for a new project instead of building on scala.rx? Understanding the principles?

Yes, mostly that, and random curiosity. The approach I'm trying is different from Scala.rx though, I guess it's closest to the "static dataflow graph" Li Haoyi mentioned in Scala.rx readme (which I've read many times). He dismisses that approach as too constrained, but I think having constraints on how you define state transitions could actually be beneficial, it might force you to separate state vs streams more cleanly in your code.

Apart from that, I just plain don't understand how Scala.rx propagation and memory management works, and there is no explanation for that in the docs other than a couple very simple examples and a mention of "best effort" (but no guarantee) to avoid redundant computation. And the tests don't seem to exercise memory management other than what's mentioned in the readme, which is not my concern.

---

I'm not sure how I feel about bidirectional operators. They're definitely useful but they break unidirectional dataflow, which is one of those constraints that I think is very useful. In react.js for example you pass down Signals (props in React terms) to child components, but child components can not write to parent's Signals (parent's state in React terms) directly, they can only expose Streams (fire callbacks in React terms). That works very nicely, even if some boilerplate is required to connect cyclical Signal (state) dependencies via streams.

Bidirectional operators feel more like Angular's two-way data binding (with the addition of project functions) which to me has proven less maintainable than react's way.

[doofin]

Are there any pure scala librarys that focus on streams/observables? In haskell world there are also similar things like conduit

[raquo]

@doofin Monix is a popular one

On Thu, Jan 11, 2018 at 9:06 AM, doofin notifications@github.com wrote:

Are there any pure scala librarys that focus on streams/observables? In haskell world there are also similar things like conduit

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/raquo/laminar/issues/5#issuecomment-356856938, or mute the thread https://github.com/notifications/unsubscribe-auth/AAggsItRvlDZakiLX0AnLNl6btbN7Z69ks5tJcEZgaJpZM4RGwMk .

[raquo]

The next version of Laminar will use Airstream instead of XStream, see #8. As that's settled for now I'll close this issue. We can still have discussions about streaming libraries or Airstream's shortcomings in new issues.