Program comparisons

[jasonkuhrt]

It would be useful at least to me to see comparisons of a program such as this

https://gist.github.com/staltz/868e7e9bc2a7b8c1f754#implementing-a-who-to-follow-suggestions-box

Written in Flyd Kifir Bacon RxJS and Elm (yes, non-JavaScript`).

It may be out of scope of this project to take on such a comparison and I would actually be interested in doing this myself at some point but since your library is the new comer to the scene I'd like to see your thoughts/effort here. Kifir in particular was among other things designed for performance https://pozadi.github.io/kefir with some demos showing that http://jsfiddle.net/jL1nm3c3/2.

Thanks for adding another FRP option in JS.

[paldepind]

Yes. Actually translating the entirety of Staltz article to Flyd could be useful.

Have you seen the examples on the readme? Two of these are my implementation of problems with RxJs equivalents.

I'm definitely adding this to my todo list.

Regarding performance: I've never benchmarked Flyd – but my initial hunch is that performance shouldn't be bad. I'm not doing anything ridiculous in the implementation. Having performance demanding examples would certainly be great as well!

[StreetStrider]

Kifir

You mean Kefir, right?

Also, Highland worth mentioning it, I might add :)

[jasonkuhrt]

@StreetStrider Corrected thank-you.

[blup]

Hi. I'd like to start by saying I find the concept behind your library incredibly simple and powerful. Before using it on a project, however, I decided to to a quick benchmark (copied Kefir's benchmark and added some test cases for Flyd). Here are a couple of quick results:

Speed

stream.map(id)
----------------------------------------------------------------
Kefir x 7,252,750 ops/sec ±1.52% (68 runs sampled)
Bacon x 1,193,492 ops/sec ±2.29% (67 runs sampled)
RxJS  x 1,190,064 ops/sec ±2.02% (65 runs sampled)
Flyd  x 167,694 ops/sec ±3.37% (61 runs sampled)
-----------------------
Kefir 1.00   Bacon 0.16   RxJS 0.16   Flyd 0.02

Memory

.map(->) (1000 samples)
----------------------------------------------------------------
Kefir   w/o subscr. 0.38 KiB   w/ subscr. +0.15 KiB   sum 0.53 KiB
Bacon   w/o subscr. 1.03 KiB   w/ subscr. +1.10 KiB   sum 2.12 KiB
Rx      w/o subscr. 0.12 KiB   w/ subscr. +0.62 KiB   sum 0.74 KiB
Flyd    w/o subscr. 3.31 KiB   w/ subscr. +3.18 KiB   sum 6.49 KiB
-----------------------
Kefir 1.00 1.00 1.00    Bacon 2.68 7.50 4.01    Rx 0.32 4.24 1.40    Flyd 8.63 21.75 12.24

.scan(0, ->) (1000 samples)
----------------------------------------------------------------
Kefir   w/o subscr. 0.44 KiB   w/ subscr. +0.15 KiB   sum 0.60 KiB
Bacon   w/o subscr. 1.23 KiB   w/ subscr. +1.48 KiB   sum 2.70 KiB
Rx      w/o subscr. 0.43 KiB   w/ subscr. +0.97 KiB   sum 1.40 KiB
Flyd    w/o subscr. 3.32 KiB   w/ subscr. +3.18 KiB   sum 6.50 KiB
-----------------------
Kefir 1.00 1.00 1.00    Bacon 2.76 9.69 4.53    Rx 0.96 6.36 2.34    Flyd 7.49 20.86 10.90

[paldepind]

@blup Thanks for putting this together! Could you please share the source? :) I'm certain these benchmarks can be improved.

[iofjuupasli]

@blup looks like code in kefir in your benchmark does nothing because of lazyness (and probably other, except flyd).

[blup]

Like I said, you have to clone Kefir's repo and adapt the tests (I only did a few). Here are the bits relevant to the actual tests: https://gist.github.com/blup/7477e4e78e8a9fdd8833 . I've included Kefir's as well for comparison.

[paldepind]

This is a real issue. But the situation can certainly improved :) I've primarily focused on clean code in the current implementation. But some of that can be sacrificed for performance. I don't think Flyd will be as fast as Kefir. But with a few changes it beats both Bacon and RxJS on the map performance test.

stream.map(id)
----------------------------------------------------------------
Kefir x 19,070,355 ops/sec ±0.09% (73 runs sampled)
Bacon x 2,077,565 ops/sec ±0.75% (72 runs sampled)
RxJS x 2,007,324 ops/sec ±0.10% (74 runs sampled)
flyd x 2,835,582 ops/sec ±0.20% (73 runs sampled)
-----------------------
Kefir 1.00   Bacon 0.11   RxJS 0.11   flyd 0.15

[blup]

Nice. Did you make some changes or is that the same benchmark?

[paldepind]

Nice. Did you make some changes or is that the same benchmark?

Yes. It's the exact benchmark code you posted. I'll be pushing a few performance tweaks later today.

[paldepind]

@blup Thanks for bringing these performance issues to my attention and for writing the code for the Kefir benchmarks.

I've just released a new version of Flyd which vastly improves performance. These are the results I'm seeing in the Kefir benchmarks for map and scan. There's quite a difference compared to the original 0.02 and the 0.15 I posted earlier. Flyd is now far ahead of Bacon and RxJS but Kefir just is really really fast! @rpominov did a very impressive job on performance in that library.

stream.map(id)
----------------------------------------------------------------
Kefir x 17,860,969 ops/sec ±0.12% (73 runs sampled)
Bacon x 2,087,587 ops/sec ±0.82% (70 runs sampled)
RxJS x 2,021,936 ops/sec ±0.09% (73 runs sampled)
flyd x 7,756,849 ops/sec ±0.11% (74 runs sampled)
-----------------------
Kefir 1.00   Bacon 0.12   RxJS 0.11   flyd 0.43

stream.scan(0, (a, x) -> a + x)
----------------------------------------------------------------
Kefir x 19,352,000 ops/sec ±0.26% (71 runs sampled)
Bacon x 1,907,498 ops/sec ±3.58% (67 runs sampled)
RxJS x 2,290,691 ops/sec ±0.15% (73 runs sampled)
flyd x 8,873,767 ops/sec ±1.21% (71 runs sampled)
-----------------------
Kefir 1.00   Bacon 0.10   RxJS 0.12   flyd 0.46

I still have a few ideas left for optimizations so expect to see another release with minor performance improvements.

And please remember that Flyd has atomic updates. This avoids unnecessary updates of streams which gives a real world performance benefit that is not reflected in these benchmarks.

[paldepind]

@jasonkuhrt I've added an implementation of the who to follow suggestions box as an example.

You can try it here and see the source code. My implementation is, however, not very close to the original one as I consider it to be overly complicated.

[jasonkuhrt]

@paldepind Thanks, some bedtime reading for tonight.

[paldepind]

Btw, I would certainly be possible to implement a version with Flyd that is closer to the original. But I like my approach. It makes it very easy to see and understand what the state is in the application and under what circumstances it changes.

[jasonkuhrt]

@paldepind I noticed that you are familiar with Elm. Is that your reference for FRP then?

[paldepind]

@jasonkuhrt Yes. That is correct :)

[jasonkuhrt]

Ah ok cool. Have you considered looking at or using the elm-runtime which already makes available FRP in JavaScript? I realize the elm-runtime is not meant to be an API in and of itself but seems to me Elm's Signal Engine (so to speak) could function like the https://github.com/Matt-Esch/virtual-dom that others can build nice APIs on top of? CC @evancz is Elm's runtime remotely close to being able to be used like that?

[paldepind]

Have you considered looking at or using the elm-runtime which already makes available FRP in JavaScript?

What would the benefit of that be? Flyd aims to be simple. The entire core implementation is less than 250 lines. Without actually having looked at the elm-runtime my guess would be that it is larger.

Since you brought it up I will mention that I don't really fancy virtual-dom. I've created a new virtual DOM library that's faster, significantly smaller and has features that virtual-dom lacks (but documentation is sparse for now). I've used it together with Flyd and union-type-js to port the Elm architecture to JavaScript. My direct JavaScript translations of the examples in the Elm architecture tutorial are available here.

[jasonkuhrt]

What would the benefit of that be?

If you are modelling yourself on Elm and Elm presumably is or is continually getting more optimized then it seems the implementation is worth at least a look. Does Elm's runtime size relate to additional features which you could do away with? I don't think its a black and white question.

Virtual DOM is an example of layering nothing more. I'll check yours out thanks.

[ntharim]

most is pretty fast too you might want to have a look: https://github.com/cujojs/most/tree/master/test/perf

Some insights into the optimisations they did: cujojs/most/issues/137

[paldepind]

@nthtran I know about most. Saying that it's pretty fast is probably and understatement ;)

I had not seen that discussion before though. Thank you for linking to it!

[jasonkuhrt]

Some insights into the optimisations they did: cujojs/most#137

Sort of interesting discussion (more like a collection of directions to follow up on) but too much machismo or whatever you prefer to call it.

[paldepind]

Since I've implemented the Who To Follow example in Flyd I guess the main force behind this issue has been resolved. Feel free to reopen if necessary.

[jasonkuhrt]

@paldepind Thanks. I have now learnt that most.js is the speed king: https://github.com/cujojs/most/tree/master/test/perf

[Mapiac]

I'm assuming that was a comparison with RxJS 4 (?) because in the Flyd post it shows Kefir faster than RxJS, but in the Most.js post it shows RxJS (5) being significantly faster than Kefir (and of course Most far ahead of both). It begs the question if RxJS 6 would be orders faster again, but from posts I saw on that project there were some concerns raised already around performance. So, given this, and that I had done experiments, and multicast is still possible with Most, yet typically it's 'cold' start and 'flyd' is always on and 'hot', would these speeds still roughly be accurate these many years ltr @paldepind