Kefir.sequentially is slow.

[xgrommx]

Hi @pozadi . I tried use Kefir.sequentially with a big collection and it's slow than Bacon.fromArray or Rx.Observable.fromArray. This is my small code:

import _  from 'lodash';
import Kefir from 'kefir';
import prettyMs from 'pretty-ms';

var items = _.range(100000);
var now = new Date();
var stream = Kefir.sequentially(0, items).map(x => x * x);
stream.onValue(x => {});
stream.onEnd(() => { console.log(prettyMs(`Kefir ${new Date() - now}`)); }
now = new Date();
... // other code with Rx and Bacon.

[rpominov]

Hi! Well no surprise, it is not the same thing as .fromArray. It emits each item asynchronously using setInterval() while .fromArray emit all of them in a loop. What are the numbers, btw? And what are you suggesting?

[Macil]

Kefir.later(0, items).flatten() would work much more closely to Bacon.fromArray.

[xgrommx]

@AgentME Oh course it's quickly. This is my code and result:

var _ = require('lodash');
var Kefir = require('kefir');
import {Observable} from 'rx';
import Bacon from 'baconjs';
import sig from 'sig-js';
import Asyncplify from 'asyncplify';
import Immutable from 'immutable';
import Streamly from './streamlyjs.js';

var prettyMs = require('pretty-ms');
var items = _.range(100000);
var now = new Date();

sig(items)
    .map(x => x * x)
    .each(() => {}).done().teardown(sig.log, `Sig ${prettyMs(new Date() - now)}`).end();
now = new Date();

Asyncplify.fromArray(items).map(x => x * x).subscribe({
    emit: (x) => {},
    end: () => console.log('Asyncplify', prettyMs(new Date() - now))
});

now = new Date();

var baconStream = Bacon.fromArray(items).map(x => x * x);
baconStream.onValue((x) => {});
baconStream.onEnd(() => console.log('Bacon', prettyMs(new Date() - now)));

now = new Date();

Observable.from(items).map(x => x * x).subscribe((x) => {

}, (e) => {}, () => console.log('Rx', prettyMs(new Date() - now)));

Streamly.fromArray = function fromArray(values) {
    var stream = new Streamly.EventStream();

    for(var i = 0; i < values.length; i++) {
        (function(k) {
            stream.onActivation(function(theStream) {
                theStream.emit(values[k]);
            });
        } (i));
    }

    return stream;
};

now = new Date();

var streamlyStream = Streamly.fromArray(items).map(x => x * x);

streamlyStream.onValue(x => {});
streamlyStream.onEnd(() => console.log('Streamly', prettyMs(new Date() - now)));

now = new Date();

var kefirStream = Kefir.later(0, items).flatten().map(x => x * x);

kefirStream.onValue(x => {});
kefirStream.onEnd(() => console.log('Kefir', prettyMs(new Date() - now)));

Result

Sig 176ms
Asyncplify 3ms
Rx 104ms
Streamly 71ms
Kefir 124ms

[rpominov]

Yep, +1 on using Kefir.later(0, items).flatten() if you need .fromArray functionality with a large array.

The exact behavior also can be created with something like this:

var stream = Kefir.stream(function(emitter) {
  items.forEach(emitter.emit);
});

But it made harder to create on purpose, as this method has issues.

To summarize:

• .sequentially is slower than .fromArray because it's a different method, and there is nothing to be done with it,
• we don't want .fromArray in API.

[rpominov]

Heh, from your results it look like Kefir.sequentially() not so much slower than Rx.Observable.from() after all :)

[xgrommx]

@pozadi Of course but it isn't Kefir.sequentially. It's Kefir.later(0, items).flatten()

[rpominov]

Ah, right, my bad. And what the result for Kefir.sequentially, just curious?

[xgrommx]

@pozadi Unfortunately result of Kefir.sequentially isn't better =(. This is code and result.

var kefirStream = Kefir.later(0, items).flatten().map(x => x * x);

kefirStream.onValue(x => {});
kefirStream.onEnd(() => console.log('Kefir', prettyMs(new Date() - now)));

now = new Date();

var kefirStream2 = Kefir.sequentially(0, items).map(x => x * x);

kefirStream2.onValue(x => {});
kefirStream2.onEnd(() => console.log('Kefir2', prettyMs(new Date() - now)));

Result

Kefir 126ms
Kefir2 1m 41.2s

Also I left comment about Bacon https://github.com/baconjs/bacon.js/issues/580

[rpominov]

Ok, pretty predictable results. 1m 41.2s ~= 100000ms, and we have 100000 items, so setInterval(fn, 0) calls the callback approximately every 1ms.

[xgrommx]

@pozadi Yes, but I think Kefir.sequentially not suitable for large collections. I'm sad =(

[Macil]

It is doing exactly what it's documented to. If you want to emit a lot of items with only occasional 1ms gaps between groups of items rather than between every item, then using sequentially, lodash's chunk method, and flatten will work well:

Kefir.sequentially(0, _.chunk(items, 100)).flatten()

[xgrommx]

@AgentME Hmm... result with Kefir.sequentially(0, _.chunk(items, 100)).flatten() more interesting but isn't better =) Kefir3 1.2s

[Macil]

It's not supposed to go faster. It's if you want it to go slightly slower. Kefir.sequentially is supposed to add delays between items.