Question: Required (code) complexity for higher-order streams

[dehmer]

Hey there!

First I'd like to express my gratitude for you guys. I find flyd to be a nice little yet powerful library to dive into the whole FRP domain.

Currently I'm playing with the idea of rewriting flyd (mostly for educational purpose.)

I noticed that updating stream values requires a quite sophisticated approach which deals with possibly creating streams recursively and evaluating/updating (possibly deferred) values in the dependency graph. Trying to justify this inherent complexity I was searching for a single (easy to express) requirement or use-case. This is what I came up with:

(Settings aside atomic updates,) higher-order streams would not be possible without support for recursively creating streams within combine functions.

While map, filter and ap for example would only need a simple recursive update algorithm, chain cannot be expressed with such a simple approach.

Can you confirm my 'suspicion'?

Your input in highly appreciated, thanks!

[nordfjord]

Hey dehmer!

The complicated approach you mentioned is there to deal with the "diamond dependency problem." It is a design goal of flyd that updates be atomic. See the relevant section in the readme here https://github.com/paldepind/flyd#atomic-updates

In practise what flyd's doing is sorting the dependencies topologically and applying updates in the order of that sort.

Hope this helps!

[dehmer]

@nordfjord Thanks for the reply. Yes efficient/atomic updates are spicing things up quite a bit. I think I'm done with my implementation, which seem's considerably easier to follow and does address a few things which caught me off-guard in flyd. When I'm done with the README/write-up I'm sharing the repository link in case you guys are interested.

Cheers!

[StreetStrider]

@dehmer

which caught me off-guard in flyd

Hello. What are those?

[dehmer]

@StreetStrider I'll let you know, once I fully documented the differences between my approach an flyd. I also intent to let flyd test cases run against my code base. At least where the "remaining" API permits it.

In the meantime, I remember one thing, which was unexpected for me: Nested evaluations are deferred until the outer evaluation has finished. At least it seems like this in the following test case.

it('nested evaluation with result', function () {
  const a = stream()
  const b = combine(a => {
    const c = stream(a())
    const d = c.map(c => c + 1, c)
    return d()
  }, [a])
  a(1); assert.strictEqual(b(), 2) // nope: b() === undefined
})

[dehmer]

@nordfjord @StreetStrider I wanted to let you know that @syncpoint/signal was just released in case you care to have a look. Any feedback would be highly appreciated, especially critique and challenging questions. Also, if you feel that I painted the wrong picture of flyd, please let my know, so I can set it straight.

I added most original flyd test cases (test/flyd.test.js), passing 50 of them. The rest is either incompatible or simply not supported.

And thank you guys for the groundwork you laid out for others to build upon!

Repository: https://github.com/syncpoint/signal

[StreetStrider]

@dehmer good to know, I will try to make a closer look. Btw, only now, when I see name signal this reminded me of mithril streams, which are almost identical to flyd, but more old. I think you are aware of them, but if not, you can also look at them and may even grab some of their tests as well, they got like 800 lines of it. Basically the same design, but never extracted into separate library.

[dehmer]

@StreetStrider Thanks! No, I wasn't aware of Mithril/streams. Implementation looks pretty minimal and straight forward. The internal structure is a little different and I like that they don't need any global state (evaluation stack in my case.) I'm gonna checkout their update strategy concerning nested streams, reads and writes.

[dehmer]

I had a good look at Mithril/streams and reduced it to its essence. Seems they only come away without global state because they don't do efficient updates. In a diamond scenario the final stream is called three times. They do atomic updates though (no intermediate values).

[paldepind]

Hi @dehmer. I'm happy to hear that Flyd has been useful to you.

I can definitely understand why certain parts of the implementation may be a bit surprising. The first version of Flyd that I wrote didn't contain any of these accommodations for updating streams within updates of streams and diamonds in the dependency graph. I would guess that this early version wouldn't have caught you off guard at all :wink:. It was only after I published the library that the feedback from other people made me aware of those issues.

You're right that updates to streams within an update to another stream are deferred. In your example, this is perhaps a bit surprising, but not deferring the updates can get very messy in other examples.

There are many interesting points in the design space of FPR and FRP-like libraries. I can see that your library also prevents glitches, which I think is crucial. If you have anything to share about your approach and thinking then you're very welcome do so :smile:

[dehmer]

@paldepind Thanks for the invite! I intend to do some more write-up, maybe even a white paper which explains how Signal is working under the hood (and why). But first I will scan flyd/issues (incl. closed) for anything I might have missed. I already saw some interesting candidates to check out.

As a side note: FRP is a field which came to my attention after I saw the expressive power it gave my own code for a pretty specific application. Digging into Solid's signals and the more recent Angular approach, I wasn't convinced by the eager push/lazy pull algorithms. Although they probably have some merits, which might be undersold. Dynamic/automatic dependency graphs (with weak refs) seem nice and all, but I don't see the need for it, aside from interfacing with DOM, which I don't care about. What bothers me most is that derivation functions are by design not pure. I would argue that this degrades expressiveness and understandability.