Streams & ADTs

[yelouafi]

Hi @paldepind, this is not exactly an issue, it's just i'm always curious about new reactive streams implementations and ideas, so feel free to close it whenever you like.

i did myself some experiments on this field, it'd be great if you could look at it (if you've got some time) and tell me what u think. This is just an experimental concept, the idea is to uses Algebraic Data Types and Promises to encapsulate the async cases.

IMO flyd seems to offer more flexibility than other libs (Bacon, Rx) when it comes to defining new operations (the self parameter on the stream callback is a simple yet powerful idea). And i like the idea of a stream being plugged like a normal callback in multiples event sources.

What's interesting is the idea of the stream end being itself a stream, but i find it quite natural from a pure functional view, the ADT definition of a list is List a = Empty | Cons a (List a), in other terms the end of list (Empty) is a list itself. So it's also natural to think of the stream end as a special case of streams (i did the same thing in my lib).

Are there any plans to support dynamic dependencies

[paldepind]

i did myself some experiments on this field, it'd be great if you could look at it (if you've got some time) and tell me what u think. This is just an experimental concept, the idea is to uses Algebraic Data Types and Promises to encapsulate the async cases.

I did a quick look. It seems interesting. I'll take a closer look later and read your blog post about it!

IMO flyd seems to offer more flexibility than other libs (Bacon, Rx) when it comes to defining new operations (the self parameter on the stream callback is a simple yet powerful idea). And i like the idea of a stream being plugged like a normal callback in multiples event sources.

I'm happy to hear that. I've made the same experiences myself. Having streams that are functions is useful when supplying them as callbacks and in any other case where a function is expected and you'd like the value to flow down a stream.

What's interesting is the idea of the stream end being itself a stream, but i find it quite natural from a pure functional view, the ADT definition of a list is List a = Empty | Cons a (List a), in other terms the end of list (Empty) is a list itself. So it's also natural to think of the stream end as a special case of streams (i did the same thing in my lib).

Having a streams ending be a stream itself was actually not my idea but @iofjuupasli's. It's a great idea however! It makes a bunch of otherwise complicated things simple.

Are there any plans to support dynamic dependencies

One could actually do that today in a module. The module would have to handle the attaching and detaching. I haven't had any such needs myself but if presented with a use case I could cook something up :)

[yelouafi]

Having streams that are functions is useful when (...) other case where a function is expected

A particularly useful case in my mind is virtual dom rendering, consuming a stream would be as simple as calling the stream function to get its actual value. which leads me to the reason for the following question

One could actually do that today in a module. The module would have to handle the attaching and detaching. I haven't had any such needs myself but if presented with a use case I could cook something up :)

i was thinking about something like automatic dependency tracking in knockout. for example

javascript a = flyd.stream(0), b = flyd.stream(0), c = flyd.computed( () => a() + b() );

`c` should detect and pick a dependency upon just calling a stream getter function. This could be useful in many cases, back to the virtual dom example

``` javascript```
var vdom$ = flyd.computed( () => h('div#c', c()) )
map( pipe(diff, patch),  vdom$ )

Another case is in for loops, for example

javascript arr$$ = ... // Stream Array(Stream) sum = flyd.computed( () => arr$$().reduce( (acc, e$) => acc + e$() ) );

Having this feature has another benefit, you can solve multiples cases without going through complex higher-order manipulations with flatMap, switchToLatest ....

In knockout this trick is achieved by having a kind of global object that maintains a global call stack. This enables a getter function to detect whether or not it was called inside a dynamic computed. So i think it'd probably require to change the core module itself in order to support this (i guess this would introduce some complications)

if you are interested you can find the details of the Knockout implementation at this link
http://www.knockmeout.net/2014/05/knockout-dependency-detection.html

[paldepind]

A particularly useful case in my mind is virtual dom rendering, consuming a stream would be as simple as calling the stream function to get its actual value. which leads me to the reason for the following question

I don't see how that helps with virtual DOM rendering. I have some examples of using Flyd with virtual DOM.

i was thinking about something like automatic dependency tracking in knockout.

Funny you should mention it. An earlier version of Flyd had exactly that feature. You can see an early documentation here. The implementation where actually quite simple. Here is how dependencies where added. It worked pretty much as you describe. The current stream was stored in a global and if another stream was accessed the accessed stream was added as a dependency to the current stream. Someone mentioned to me back then that it looked a lot like Knockouts computed values.

As you know the feature is no longer there. I took it out since after doing a bunch of programming with Flyd it turned out to not be of much use.

In this earlier version of Flyd you didn't have to specify a streams dependencies. You could do something like this:

var sum = stream(function() {
  return x() + y();
});

And Flyd would automatically figure out that sum depends on x and y. There is one catch though. You can only figure out the dependencies of a stream by actually invoking it's body. So if sum was declared before either x or y had a value the initial value of sum would be NaN. Thus it was also possible to manually specify a streams initial dependencies to ensure that it was not invoked before they had values. That limitation turned out to be pretty much everywhere when writing general functions on streams. So the automatic dependency resolution wasn't used much and I decided that is wasn't worth enough to keep it.

Having this feature has another benefit, you can solve multiples cases without going through complex higher-order manipulations with flatMap, switchToLatest ....

How do you think having automatic dependency tracking would help implement those functions?

I've read most of your blogpost and there are certainly things I like the look of! I'll write a comment when I've read it all and had time to think about it :)

[yelouafi]

I don't see how that helps with virtual DOM rendering. I have some examples of using Flyd with virtual DOM.

I was rather thinking of something like this (taking the counter example)

inc$ = stream()
dec$ = stream()
model$ = merge( inc$.map(1), dec$.map(1) ).scan(sum, 0)

view$ = stream( () =>
  h('div', {style: countStyle}, [
    h('button',{ onclick: dec$ }, '–'),
    h('div', {style: countStyle}, model$(),
    h('button',, { onclick: inc$ }, '+'),
  ]))

If there are many streams referenced from view$ then it'd be awkward to list all dependencies in parameters. Automatic tracking can then simplify the view code.

So if sum was declared before either x or y had a value the initial value of sum would be NaN

I'm not aware of the internal bits of flyd, but can't we just check if all detected dependencies have or not yet a value before deciding to assign the function result to the stream?

How do you think having automatic dependency tracking would help implement those functions?

Actually i was thinking of getting rid of them. Not definitely, but in many cases involving complex dynamic state. Take for example this requirement from the TodoMVC app,

The "Mark all as complete" checkbox should also be updated when single todo items are checked/unchecked.

So the toggleAll checkbox state must reflect the toggle state of all todo items. How do we represent this state using simply higher order functions like flatmap or switch ? I can't think of a simple way, but using dynamic collections and automatic dependency tracking it's easy

todos$ = ....
isAllDone$ = stream( () => todos$().every( t$ => t$.isDone$() )

I've read most of your blogpost and there are certainly things I like the look of! I'll write a comment when I've read it all and had time to think about it :)

Thanks !! I'm looking forward for your comments.

[paldepind]

If there are many streams referenced from view$ then it'd be awkward to list all dependencies in parameters. Automatic tracking can then simplify the view code.

Then you would probably pass an object to the view containing all the streams. But dependency tracking can't be used here anyway. The view doesn't depend on your inc$ or dec$. It only sends values to them.

I'm not aware of the internal bits of flyd, but can't we just check if all detected dependencies have or not yet a value before deciding to assign the function result to the stream?

In some cases that would work. In this case it would because ending up with NaN is not a fatal error. But what if the stream expects an object from one of its dependencies? Then trying to access that object would throw an error.

So the toggleAll checkbox state must reflect the toggle state of all todo items. How do we represent this state using simply higher order functions like flatmap or switch ?

If we had a stream of all todos we could do this:

isAllDone$ = flyd.map(todos => todos.every(t => t.isDone));

I can't think of a simple way, but using dynamic collections and automatic dependency tracking it's easy

I don't think I get the point of your example. But I'm not personally in favor of the approach anyway. I think trying to model all state in your program with streams like that is a bad idea. I went down that road initially when using Flyd and I found that the complexity of my applications increased. I now prefer to model my state with as few streams as possible. I've actually implemented TodoMVC with Flyd here. I use 4 streams for the entire application. The entire state of the application is inside a single stream called model$. This has many advantages. One important one is that all the functions that operate on the model can be normal pure functions that doesn't have to know about streams.

[yelouafi]

The view doesn't depend on your inc$ or dec$. It only sends values to them.

No, in the above example the only dependency was model$.

isAllDone$ = flyd.map(todos => todos.every(t => t.isDone));

This will not work if t.isDone is itself a stream.

I don't think I get the point of your example. But I'm not personally in favor of the approach anyway. I think trying to model all state in your program with streams like that is a bad idea.

I've read the examples, i know you're actually using the same pattern as Elm : a big static model, actions as ADTs (union types), a single channel to collect view events, effectively restricting the use of streams to a single big scan that keeps invoking the update function on each new event.

To be honest, i'm not very fond of this method, it feels like w're replicating again to the conventional imperative paradigm: Listen to events -> Update state; The original goal of FRP was to represent a state as a dynamic value (behavior) and then model more complex states by composing existing behaviors. Behaviors are meant to be composable abstractions, and composability enables scalability to an arbitrary degree of complexity.

IMHO the Elm architecture is just a consequence not a choice. Elm lacks support for dynamic networks (AFAIK), which makes it complex or even impossible to model complex states by just composing other signals/streams. I think this is a recurrent problem when you try to unify events and state in a single abstraction. Normally, an event has a value defined only in discrete moments of times, whereas a state (behavior in FRP) has a value defined in every moment of time. This is the semantic model of the original FRP formulation made by Conal Elliott.

Having automatic tracking enables us to express the essential semantics of the original FRP : modelling complex and composable states. But for this we need the behavior semanics: ie a state that have a value defined in each moment. I think this the essence of the problem you have back with automatic tracking and streams that have not yet started to produce values

[paldepind]

No, in the above example the only dependency was model$.

Yes. But you wrote this:

If there are many streams referenced from view$ then it'd be awkward to list all dependencies in parameters.

Which doesn't make much sense because the view will always only depend on a single stream.

I've read the examples, i know you're actually using the same pattern as Elm : a big static model, actions as ADTs (union types), a single channel to collect view events, effectively restricting the use of streams to a single big scan that keeps invoking the update function on each new event.

Exactly.

To be honest, i'm not very fond of this method, it feels like w're replicating again to the conventional imperative paradigm: Listen to events -> Update state;

The state depends on events. I don't see how modeling it as such is a bad approach. And this still gives us a lot of advantages compared to imperative code.

• Reactive one way event flow – this makes it easy to figure out how state changes
• Pure side effect free functions with all the benefits that brings
• State inside a single objects gives many benefits. State is easy to inspect, easy to persist, easy time travel.

The original goal of FRP was to represent a state as a dynamic value (behavior) and then model more complex states by composing existing behaviors. Behaviors are meant to be composable abstractions, and composability enables scalability to an arbitrary degree of complexity.

Sure. Streams in Flyd are composeable as well. And that certainly allows for arbitrary scalability. But that is not the same as saying that is scales in a simple way.

IMHO the Elm architecture is just a consequence not a choice. Elm lacks support for dynamic networks (AFAIK), which makes it complex or even impossible to model complex states by just composing other signals/streams.

Elm's Signals are not monadic. And Elm only supports static dependencies which means that the entire dependency graph can be known from the beginning. But this is not a consequence – it is a choice. Monadic behaviors and events where known when Elm was created. Not including it was a conscious decision.

I think this is a recurrent problem when you try to unify events and state in a single abstraction.

I don't see a connection there. Flyd only has streams/signals that updates in discrete steps but they are still monadic.

Normally, an event has a value defined only in discrete moments of times, whereas a state (behavior in FRP) has a value defined in every moment of time. This is the semantic model of the original FRP formulation made by Conal Elliott.

I'm not sure what you mean with normally here. Flyd does not the implement the exact same semantic model of classic FRP. That is by intent. Flyd implements a model quite similar to Elm. A stream in Flyd is defined for every moment in time but it only changes in discrete steps. It has no concepts of continuous time.

Having automatic tracking enables us to express the essential semantics of the original FRP

Original FRP did not have a concept of automatic dependency tracking.

But for this we need the behavior semanics: ie a state that have a value defined in each moment.

We don't need behavior semantics for that. That would be achieved simply by enforcing that all streams should have an initial value.

There a many problems associated with trying to model everything as separate streams/behaviors. Whenever you load something from a backend you will have to initialize streams based on it. And when you need to send data back again you'll have to convert the nested streams to JSON. You can't as easily represent you view as a single function on your state like in the Elm approach.

You'll end up with streams inside streams inside streams. There is no end to it. You'll have to do a lot of unwrapping/flattening to get the values you need. Suddenly the dependencies between the state in your program becomes tangled. There is no longer a straightforward unidirectional path through the state.

Allowing automatic dependency tracking would make all of this even more complex. Then dependencies are suddenly implicit instead of explicit. Dependencies can change uncontrolled. Instead of using known functions on streams you define ad-hoc dependencies.

I've tried this approach before turning to the Elm approach. I created flyview which instead of taking the virtual DOM approach was based on the idea that every changing value in the view could be hooked up to a stream. I.e. everything was represented as streams. I experienced all the problems I mentioned above.

Om btw. takes a similair approach as Elm.

I'd like to hear what you think about the above :) Btw, I like how the function definitions on reactive values in your library mirrors the same functions on lists.

[yelouafi]

The state depends on events. I don't see how modeling it as such is a bad approach. And this still gives us a lot of advantages compared to imperative code.

• Reactive one way event flow :+1: : this is not a characteristic of just Elm-style apps only, but of all data-flow architectures.
• Pure side effect free functions :+1: : same as above, you can implement business logic in pure functions and lift them to work with reactive values.
• State inside a single objects gives many benefits: sure, but i'll trade that for a conventional debugger if it that means loosing composability

But this is not a consequence – it is a choice. Monadic behaviors and events where known when Elm was created. Not including it was a conscious decision.

I don't think so, from the Elm architecture site

I actually discovered this pattern just using Elm and have been shocked by its simplicity and power.

But my real point was that with just 1st order signals you have no other choices to write scalable apps.

A stream in Flyd is defined for every moment in time but it only changes in discrete steps. It has no concepts of continuous time.

I'd rather distinguish :

1- function defined over continuous time 2- function that varies smoothly over continuous time 3- function that varies discretely over continuous time

as 2 et 3 are just special case of 1

If i'm not mistaken, a stream in flyd doesn't necessarily start with a defined value right away, wasn't this the reason of the hasVal property?

a = flyd.stream();
b = flyd.stream();
c = flyd.stream([a,b], function() {
  return a() + b();
})

a(10) // a has a value, but b and c not
b(20) // now b and c each have a value

I think my question here is what is the type of a stream getter function get : (stream a) -> ? ?

• If it is a then a stream has to return a value of type a whenever it's invoked
• If it is Maybe a then a stream can return either a a or undefined at any point,

but this is not the case here, (again if i'm not mistaken) a stream initially starts with Maybe a type but once it has a value it'll always returns a; this seems to me quite similar to a Promise meaning.

Original FRP did not have a concept of automatic dependency tracking.

Perhaps i would say : Having automatic tracking enables us to expressimplement the essential semantics of the original FRP.

I don't see a connection there. Flyd only has streams/signals that updates in discrete steps but they are still monadic.

My point, is once you gather events and behaviors in a single abstraction you automatically ends up with a type signature Signal a : Maybe a and then you'll have to think only discretely (something similar has been explored by Paul Hudak in this paper ). As a consequence, you have to deal with the problem of absence of value.

but a discrete model is only an approximation to the continuous one, and the first can't reach the full expressiveness if the second. Monadic streams can be used to express some complex dynamic functional dependencies between behaviors to a certain degree, but with the cost of, as you have said, of an increasing complexity.

We don't need behavior semantics for that. That would be achieved simply by enforcing that all streams should have an initial value.

this is not possible if the stream carries the meaning of both events and behaviors, what is the start value of the increment stream from the counter example?

you will have to initialize streams based on it. And when you need to send data back again you'll have to convert the nested streams to JSON

I don't see how this issue is specific to a purely dataflow approach. Even in Elm-style you have to init your models.

You'll end up with streams inside streams inside streams. There is no end to it. You'll have to do a lot of unwrapping/flattening to get the values you need. Suddenly the dependencies between the state in your program becomes tangled. There is no longer a straightforward unidirectional path through the state.

I think it's all matter of code organization. the way i see it is just like with components in an electrical circuits. you split your parts in different components. a component is just function that take signals in input and provide a set of signals as output. This represents the interface/contract of the component. We can wire different components together to form higher order components and so on. We still have the unidirectional dataflow and you can have a clean global and dynamic view of your system interactions

[paldepind]

I don't think so, from the Elm architecture site

Yes. You're right that the Elm architecture is a consequence. But I was talking about what went before that: the decisions to not have monadic signals and thus to have a static dependency graph. Those we're conscious choices and they lead to the Elm architecture. So saying that "the Elm architecture is just a consequence not a choice" is unfair since it is a consequence of something that was a choice.

If i'm not mistaken, a stream in flyd doesn't necessarily start with a defined value right away, wasn't this the reason of the hasVal property?

Yes. That is correct. Your additional remarks are as well. But when one reacts reactively to changes from streams their potential initial undefinedness is not noticed.

this is not possible if the stream carries the meaning of both events and behaviors, what is the start value of the increment stream from the counter example?

If you define a stream with an initial value you can consider it's getter to be of type Stream a -> a. If you on the other hand do not define it's initial value (like with increment) you can think of it as Stream a -> Maybe a. But in that case you aren't interested in its getter anyway (why would you want to get an old increment event). Thus I think you can model something similar to both events and behaviors.

Perhaps i would say : Having automatic tracking enables us to expressimplement the essential semantics of the original FRP.

Can you elaborate on how that brings us closer to the "essential" semantics of the original FRP. And btw. you've not said what you consider the essential semantics of FRP. What parts of the original semantics of FRP do you consider non-essential?

I don't see how this issue is specific to a purely dataflow approach. Even in Elm-style you have to init your models.

I think it's all matter of code organization. the way i see it is just like with components in an electrical circuits. you split your parts in different components. a component is just function that take signals in input and provide a set of signals as output. This represents the interface/contract of the component. We can wire different components together to form higher order components and so on. We still have the unidirectional dataflow and you can have a clean global and dynamic view of your system interactions

Let's say you receive JSON from a backend like this: {type: 'rabbit', name: 'Thumper', mother: {type: 'rabbit', name: 'Bella'}}. These values can change and if you represent everything as streams you'll have to recursively convert all the JavaScript primitives to streams. It is doable for sure. You could use flyd-obj. But it is complex. With the Elm architecture you don't need to convert anything to streams – you just inject it into the state.

Consider another example: you have a stream of a list of todos where each todo contains 4 properties that are also streams. You also have a stream of the index of the currently selected todo item. Now think about how you would create a stream representing whether or not the currently selected todo item is done. I.e. the property would depend on the list of todos, the index, and the status of the selected todo. It is certainly do-able. But you would have to do a bunch of lifting and flattening. With the Elm architecture your state is just plain JavaScript data types so you can just do selectedStatus = todos[selectedIndex].active. That is much simpler. You cannot remove that complexity by "code organization".

FRP is awesome when used in controlled. But it can easily be overused. I've tried modeling everything as streams. It looks neat at first but suddenly a significant part of your code is just putting all the streams together. You have to maintain and look at all the "electrical circuits" and the "wire" as you put it.

On another note: I've read your blog post. The "beautiful" in the title is not an understatement. The approach really is beautiful and elegant. Offloading a lot of the work to promises is a clever idea. Flyd is implemented more like traditional event listeners but expanded so it can express streams. I really like the approach. It certainly is more beautiful than Flyd in some aspects (though I think Flyd wins in other). The API is way too object oriented IMO. And I don't think you can implement atomic updates with that approach.

[yelouafi]

So saying that "the Elm architecture is just a consequence not a choice" is unfair since it is a consequence of something that was a choice.

I think we agree both on the same thing.

With the Elm architecture your state is just plain JavaScript data types so you can just do selectedStatus = todos[selectedIndex].active. That is much simpler. You cannot remove that complexity by "code organization".

You just gave the answer youreself

selectedStatus = computed( () => todos()[selectedIndex()].active() )

Code organisation is meant for the case where you've a high number of streams/signals. Grouping them into components simplifies the reasoning about dependencies in the system

Can you elaborate on how that brings us closer to the "essential" semantics of the original FRP. And btw. you've not said what you consider the essential semantics of FRP. What parts of the original semantics of FRP do you consider non-essential?

Essential FRP semantics for me are those relevant to GUI apps : Continuous time + Declarative reactivity. For others it may be another thing (integration, instantaneous predicate events, time transformation...).

There is a common misunderstanding about continuous time in FRP, people often think it's about animations. As i understand it, it's much more basic than that: it simply means that your signals are, semantically, functions defined over a continuous domain. Whether those functions varies smoothly or discretely or even stay constants is just a special case.

For example say we have these 2 signals a and b, then if we define a signal c = t => a(t) + b(t) then the FRP system must ensure that the relation c = a + b holds in every moment. It doesn't matter if a and b are constants, reactives or smoothly animated.

Now if you think that the above example is simple as c = stream([a,b], () => a()+b() ) then let me introduce a slightly more complex examples.

we have a signal s : Signal [Signal Number]. How do get the sum of the nested array of signals. You'll likely start to think of a strategy involving higher-order/monadic manipulations. In the continuous FRP model the solution is derived directly from the problem

// array$ : Signal [Signal Number]
sum$ = t =>  array$(t).reduce( (acc, n$) => acc + n$(t) )

We can write the above code because we know that our FRP system wil maintain the functional dependency between sum$ and array$ at every moment of time. I'm not just talking about writing simpler code, the difference, to me at least, is more fundamental than that: the continuous model enables us to derive the solution as soon as w've finished describing the problem. In a discrete model, involving streams, you`ll have to translate the problem in terms of discrete list-like manipulations: merge, scan, flatten.

Automatic dependency tracking helps achieving the above expressiveness because it enables writing siganls/streams with dynamic dependencies: ie the list of dependencies is itself a signal (BTW this seems to be an interesting idea sum$ = stream( array$, () => .... ) can be this implemented in flyd? instead of providing an array of deps, provide instead a stream of deps).

In flyd, you get the current value of a stream s by s(). If you think of the stream as a list, then the meaning of current value is the last element of that list. Now if you think of a stream as a function of time, you can view s() as equivalent to s(Date.now()). And you can think of flyd as a kind of a continuous FRP system with restrictions placed on values of parameter t. If you add to that the ability to specify streams with dynamic dependencies, then IMO you are close to the semantices above.

The 2nd of essential semantics is Declarative reactivity it means that signals can be a function of event occurrences in addition to time. the original FRP had a until operator that works like this

// one time switch
sig = sig1.until( next('click').then( sig2) )

//or, recurrent switch
sig = sig1.switch( each('click').then( sig2 => sig2) )

which means sig behaves like sig1 until the next click than acts like sig2, the second version is a recurrent switch, just like switchToLatest

In contrast, with traditional observer pattern

var s = s1;
// observe one time
sig = once('click', () => s = s2 )

//observe all 
sig = on('click', e => s = computeValFromEventAndCurrentState(e) )

The main benefits of the first approach IMO

1- it's composable, sig is a first class value that can be combined with other signal, themselves reactive to other events,

2- it's more deterministic: in the imperative observer way, program is a kind of asynchronous state machine that steps on each event by combining the event input and existing states; states machines are difficult to reason about in complex event-driven apps mostly for the reason of order. You've to ensure that you state mutations are done in the correct order, this approach is error prone and many bugs remain hidden behind edge cases. In the FRP approach, order of things is automatically preserved thanks to the functional dependency.

On another note: I've read your blog post. The "beautiful" in the title is not an understatement. The approach really is beautiful and elegant. Offloading a lot of the work to promises is a clever idea. Flyd is implemented more like traditional event listeners but expanded so it can express streams. I really like the approach. It certainly is more beautiful than Flyd in some aspects (though I think Flyd wins in other). The API is way too object oriented IMO. And I don't think you can implement atomic updates with that approach.

Thanks !! happy you liked it. the pure FP approach lead to more expressiveness but has also many drawbacks, if you put my lib in any benchmark i'm pretty sure i'll be the last one. There are also some memory subtleties if you're not careful enaugh

[paldepind]

You just gave the answer youreself

That is not viable answer to me. Then you've just made the dependencies and the flattening implicit and hidden the complexity away in the automatic dependency resolution. This is like wanting to model everything as a stream. Just look at the function you wrote. Where is the plain pure function that just takes input and return a value? It is gone. And that is a bad thing. Remember what you wrote earlier: "you can implement business logic in pure functions and lift them to work with reactive values.". Now you've abandoned that idea and pure functions with it as well. But pure functions are amazing! They are easy to understand, easy to test and easy to refactor. What you're doing is similar to automatic lifting of functions. Here are some arguments against that.

we have a signal s : Signal [Signal Number]. How do get the sum of the nested array of signals. You'll likely start to think of a strategy involving higher-order/monadic manipulations. In the continuous FRP model the solution is derived directly from the problem

How can you claim that continuous FRP helps solve the problem and then immediately after show the solution you want with Flyd+depency tracking even though that is not continuous. What you think makes the solution better is not continuous time – it is automatic dependency tracking. Something that classic FRP did not have. Your sum example doesn't do continuous time justice either. You showed it yourself: Flyd can easily solve it today. You're examples or arguments doesn't really explain the benefits of continuous time. And you've not explained how automatic dependency tracking brings us closer to the original FRP semantics.

One case where continuous time actually helps is when you have a stream of velocity and wants to get a stream of distance covered. Then you can simply do distance = scan(dist, v => dist + v, velocity). The semantic model of classic FRP makes it very easy to define integrals of behaviors. Since because behaviors are just functions of time you can do the same things with them as you can with normal functions (like integration). It generally allows one to think at a higher level when composing streams. But implementation is significantly more complex. You for instance have to do implement numerical integration.

Behaviors are just not attractive to me. I see much more use in streams as a list of values over time. You're FRP implementation shows very well how elegant such a concept is. Overall I consider that approach more practical for solving every day problems in JavaScript.

The 2nd of essential semantics is Declarative reactivity it means that signals can be a function of event occurrences in addition to time. the original FRP had a until operator that works like this

Implementing the until operator in Flyd is certainly possible.

states machines are difficult to reason about in complex event-driven apps mostly for the reason of order. You've to ensure that you state mutations are done in the correct order, this approach is error prone and many bugs remain hidden behind edge cases. In the FRP approach, order of things is automatically preserved thanks to the functional dependency.

Can you expand on that?

[yelouafi]

Then you've just made the dependencies and the flattening implicit and hidden the complexity away in the automatic dependency resolution

This just seems a good thing to me.

This is like wanting to model everything as a stream.

Yes, this is the whole prupose of FRP and the Dataflow paradigm in general

Just look at the function you wrote. Where is the plain pure function that just takes input and return a value? It is gone

I think this goes as well for a simple call val = someSteam(). So this is not specific to automatic tracking (AT). But anyway, as i said consider stream getters as taking an implicit value of Date.now() and then you get pure functions.

Here are some arguments against that.

if i understand well, the main point of Evan against monadic signals are:

considering the case of a higher-order signal expression, like a stateful signal that switches, recurrently, between 2 other stateful signals (themselves accumulating from 2 events):

If we consider that the 2 nested signals aren't updating when not used then the value of the resulting signal becomes non determinsitic as it is not fully defined by the inputs signals but also with the time when the signal is active (switched to). Conversely, if the signals keep updating from the time when they have been created , then w've to determine if those signals are created from the start time of the program or lazily from when they get first evaluated. in the first case, either we have to keep all the history of the inputs signals, which lead to serious memory issues. In the second, if we have 2 expressions with the same expression addClicks(x,y) = count Mouse.clicks + x + y evaluated at 2 different points in time w'll have 2 different results from the same expression.

i'm a bit uncomfortable with the mixing of denotational and operational issues here (denotational issues=non-deterministic/non-pure values, operational issues = signal updates, start of evaluation...). If we break things apart, w'll start with the fact that events/discrete signals are nothing more than a list of pairs (time, value), possibly infinite but with a well defined start time. then as a consequence yes, signals update themselves when unused (they need to in order to comply the semantics of list scan, we don't miss any element in the list).

Now w'll have to make explicit in our expressions the time parameter (w're in the denotational world), and the expression becomes addClicks(x,y,t) = count Mouse.clicks.at(t) x y. ie. we're slicing the Mouse.clicks stream at t. But, slicing from what position? So w're also asking when the count accumulator function starts counting the mouse clicks, from the creation time or evaluation time of the addClicks

Evaluation time is a non denotational concept but tied to a specific execution model (lazy evaluation models), worse it's also non-deterministic or at least very-hard to reason about (undecidable like Evan says) because it depends on the entire execution path that led us to force the evaluation. So w'll discard it.

So signals starts updating from the creation time, but then w'll have as Evan says a problem of the same expression sig(t) = addClicks(x,y,t) = count Mouse.clicks.at(t) x y returning different results depending on when the signal sig has been created so w'll nomore have pure function (this is a really serious issue if encountered in the denotational phase).

But in the expression above, We're supposing that the event stream Mouse.clicks starts with (ie has as first element) the time after sig has been created. But where is the reference to this parameter? w'll have to make it explicit in order to clean up our denotational model, so the expression becomes now sig(t0, t) = count Mouse.clicks.from(t0) x y and we nomore have the non purity issue.

Of course there is still the space/time leak issue, and this is as Evan said, ' It is a common and very long discussion in real life' and an exhausted debate (if FRP could speak it would say: this is my life story :)). And those problems are inevitable when try to write a pure FP implementation of FRP**, pure FP implementations of FRP are necessary in order to ensure that the result is conform to the denotational model, as in non-pure implementations (mutable state), we can end up with non deterministic values due to concurrency .

But as we know, JavaScript is a single threaded environment, and in a single threaded env. we can have a deterministic view of the state because there is no non-deterministic interleaving of threads that updates the same state. And as i said, if you consider the time parameter implicitly Date.now() always when you call a aStream(), you can reason about your program from a purely denotational view.

Also, Because also JavaScript is a strict language you have a deterministic value of the famous start time. You're probably aware of issues introduced with lazy evaluations in libraries like Bacon, From what i said above, i consider those are not simple bugs, they are really issues on the underlying semantical model, since the start time parameter depends on the first time of evaluation, which can be undecidable as Evan said.

How can you claim that continuous FRP helps solve the problem and then immediately after show the solution you want with Flyd+depency tracking even though that is not continuous

I never said flyd has a purely discrete model. It's more an hybrid one. Consider the following example

a$ = stream(), b$ = stream(), c$ = stream()
d$ = stream([a$, b$], () => a$() + b$() + c$() )

clearly this is wrong, c$ is called even though it's not a dependency for d$. But this rule of calling from a body function only streams that are listed in the dep. list is not enforced by the lib. In fact what gives the stream the behavior semantics is that i can call the getter at any moment, if i can do so then the getter function is defined over a continuous domain. In other terms, it is a behavior.

If w're not allowed to to do the above, then w've to drop the getter function and access only values from inside the body functions

a$ = stream(), b$ = stream(), c$ = stream()
d$ = stream([a$, b$], (a,b) => a + b )

Then we have pure discrete semantics and forget all i things i said about automatic tracking.

Now, if w're allowed to call the getter function at any time, then you have to make a distinction between stateful/continuous signals (behaviors) and discrete ones (events). If w've not been provided a start value it'll be a discrete one and there is no sens to have a notion of current value. otherwise we've a stateful signal whose value can be sampled at any moment (Bacon has the same concept EventStream vs Property).

You're examples or arguments doesn't really explain the benefits of continuous time

i feel like there is a confusion here between continuous function (defined over a continuous domain) and continuous function that varies continuously over its continuous domain. conitnuous time means you can sample you signal at any moment and you`ll get the exact value defined by its functional dependencies.

What you think makes the solution better is not continuous time – it is automatic dependency tracking. Something that classic FRP did not have.

I still see the same confusion denotational/operational here and above. FRP does not have automatic dep. trackin, neither any other denotaional model. Denotational models are about defining what things are, ie. define their values in terms of other well defined things using pure/math functions. While operational models try to answer the 'how', ie. implements semantics of the denotational model on a certain execution environment (machine) as a steps of statements.

a denotational model says for example a(t) = b(t) + c(t). he dont care how this is achieved, what matters here is the denotation, the meaning or the pure/functional definition. It may be achieved by listeners, automatic tracking, dep. network, directed graphs... but all this is irrelevant to the denotational model. So saying that FRP doesn't have automatic tracking is mixing 2 different worlds of view.

And you've not explained how automatic dependency tracking brings us closer to the original FRP semantics.

AT helps us implement a subset of the continuous semantics. It's a purely push (operational) approach that allows us to overcome the constraint of explicitly listing the dependencies when writing behavior expressions.

Can you expand on that?

when you write expression like

a = t => ..., b = t => ...
c = t => a(t) + b(t)

If you call c(someT) you're sure that expressions will get called/evaluated in the correct order, functional dependency automatically maintains the order for us.

But in the imperative way, you'll have to ensure yourself the order of things, if you do

// at some time
a = ...,
c = a + b
b = ...

then things go wrong because you missed the correct order of things. Note that in the case above even if you're dealing with immutable data, the problem still persists because it relates to the sequence of imperative instructions you're issuing.

In the case of simple apps when an event lead to a single, or even a few more, imperative push it may be no a problem. But in complex event driven programs there can be a veritable nightmare to maintain, especially in multithreaded environments due to race conditions. Of course, JavaScript is single threaded but there are still many cases like that ((Just ask the Node.Stream developers, or even better look at Stream#pipe method source code)

[paldepind]

Thank you for your reply. It was an interesting read!

i feel like there is a confusion here between continuous function (defined over a continuous domain) and continuous function that varies continuously over its continuous domain. conitnuous time means you can sample you signal at any moment and you`ll get the exact value defined by its functional dependencies.

I think I misunderstood you before. Your point is that in Flyd a stream with an initial value is defined over a continuous domain (time) but that it does not vary continuously over time?

then things go wrong because you missed the correct order of things. Note that in the case above even if you're dealing with immutable data, the problem still persists because it relates to the sequence of imperative instructions you're issuing.

Thank you for elaboration. I see what you mean. It could certainly be an issue in a realistically sized application. In your example the relationship between a and b could be automatically maintained by FRP.

I think your arguments are convincing. I'm very close to go ahead and re-implement automatic dependency tracking – at least in a separate branch! But I still see one issue. Most other JavaScript reactive libraries implement lazy streams because it gives automatic resource management. Just in case your not familiar with it there's an explanation here. It allows for easier garbage collection. But this form of laziness results in certain issues as you noted yourself:

You're probably aware of issues introduced with lazy evaluations in libraries like Bacon, From what i said above, i consider those are not simple bugs, they are really issues on the underlying semantical model, since the start time parameter depends on the first time of evaluation, which can be undecidable as Evan said.

Until now this easing of memory management has not been a problem to me because I use Flyd with mostly static stream/signal graphs. I.e. without using the monadic properties of streams. But the style that automatic dependency tracking would allow would make it very easy to dynamically create a lot of streams and thus make it very easy to introduce space leaks. What do you think about that issue?

[paldepind]

@yelouafi

A particularly useful case in my mind is virtual dom rendering, consuming a stream would be as simple as calling the stream function to get its actual value. which leads me to the reason for the following question

If one uses streams to the amount that you're suggesting then virtual DOM gets unnecessary. flyview was founded on the idea that if every changing state is a stream then we can just hook up the DOM to these observable streams.

With flyview you can create a header like this:

v('h1', name$),

And since name$ is a stream the header will automatically update when name$ does. You can also do stuff like this:

v('ul', {class: 'sub-tree'}, [
  flyd.map(sel => sel ? activityView(actions, leaf) : null, leaf.selected),
]),

This will include the view returned by activityView in the <ul> whenever the stream leaf.selected is true.

[yelouafi]

Your point is that in Flyd a stream with an initial value is defined over a continuous domain (time) but that it does not vary continuously over time?

Exact.

Until now this easing of memory management has not been a problem to me because I use Flyd with mostly static stream/signal graphs. I.e. without using the monadic properties of streams. But the style that automatic dependency tracking would allow would make it very easy to dynamically create a lot of streams and thus make it very easy to introduce space leaks. What do you think about that issue?

If i understand well: subscription from an observable to its source(s) is delayed until the stream or one of its descendants is subscribed to with a terminal method onValue. in which case the whole chain is activated up to the root source. and the argument is that if there are no terminal subscriptions, then references to dependents are released which makes it possible to garbage collect a stream once any other program-specific reference to it is gone. So the trick is to always unsubscribe with an ofValue when \ we no longer need the stream.

I feel the same confusion Denotational/Operational here. We are trying to resolve a non functional requirement based on ambiguous semantics. in the example of window scroll event, we deliberately miss scroll events in order to allow efficient memory managament. I think the main problem with the above reasoning is that the library authors are not explicit with their underlying semantics. You've no choice, you'll have to be explicit and formal with your expected behavior (ie have a denotational model). Otherwise you're trying to build your implementation and optimizations on top of something undetermined. So i think that the above approach doesn't make quite sens to me.

Now, if we choose to not miss any stream event, w'de have to resolve potential memory issues, because streams always hold references to their dependents. we can no longer free a stream by just dereferencing it from our program. So our main issue here is about those ties in the dep. graph that prevents a stream from falling from the tree.

I think the key concept here is "time". Memory management is about the lifecycle of objects, so w'd have to reason about 2 times: the creation time of a stream and the time when it can be freed. But time also is a key concept in the underlying semantical model, because event streams must have a well defined start/end time.

In order to comply with the before-mentioned semantics of lists, a stream has to start when it's created; but in order to fullfill his complete semantics, it has also to stay alive (in memory) until it yields all the expected occurrences.For example, if i say stream = mouseClicks.timeout(10 secs) then stream denotes a sequence of all user mouse clicks until 10 seconds from now. This implies that stream must stay in memory until it yields all the required events (note that the start time is always implicitly Date.Now so it's the same as if w're doing MouseClicks.from(now).to(10 secs after)). The dependency model used in flyd, ensures this because the dependency chain up to the source (eg. a DOM element) prevents the stream from being garbage-collected.

in the example of Kefir discussion above stream = Kefir.fromEvents(window, 'scroll').map(...).filter(...).take(...). The term Kefir.fromEvents(window, 'scroll')semantically means that w're creating an infinite stream because w're not explicit about when the stream ends. during execution, infinite streams means they'll stay alive at least until the original source (eg. a DOM element like window above) is garbage collected.

Now the question is about the relation end time/garbage collection time. In the flyd dep. model. obviously the 2nd has to be after the 1st. when a stream ends, it means it'll stop issuing events so there is no more need to notify its dependents, so we can cut the dependency ties to its descendants . And conversely, if a stream ends and stops issuing events then there is no more need to update it so it can also be removed from the dep. list of its sources.

To resume when a stream ends, we have to

1- empty its list dep. list (if i'm not mistaken, flyd does already this in endStream) to set free the dependent stream. 2- Remove the stream itself from the dep. lists off all its sources, since it'll not be updated anymore.

Unless i miss something, the 2 above rules will enable a stream to be garbage collected by simple deferencement. And better than the lazy model above, there is no need to imperatively unsubscribe from a stream by manually calling ofValue. this can be done automatically by the library thanks to the time relations established above.

A final note about those infinite streams created from DOM elements, i said above that the stream will be freed once the DOM element itself is garbage collected. But in cases of event delegation listeners are registered in some parent element. In this case the event module has to ensure removing all listeners targeting a child DOM element once this one is removed from the DOM tree (this is the right time to miss events since once reomved from the DOM an element will receive no events). it has to be done anyway, using streams or not, otherwise the GC will be unable to collect it so this a general rule not just specific to frp.

I hope i'm not missing something, the above was elaborated a bit quickly so maybe there are some flaws in my reasoning.

If one uses streams to the amount that you're suggesting then virtual DOM gets unnecessary.

yes, you're right. The push model propagates changes through the system. Thus we can infer DOM updates from those changes. A little issue however is presented by dynamic collections, if i've a stream representing a dynamic array, all i get from the model is the old and the new array, but no detailed infos. on the change that happened (insertion, removal, reorder or a combination of many of them) so 'ill have to do some ad-hoc synchronisation just as we do in virtual DOM

[paldepind]

If i understand well: subscription from an observable to its source(s) is delayed until the stream or one of its descendants is subscribed to with a terminal method onValue. in which case the whole chain is activated up to the root source. and the argument is that if there are no terminal subscriptions, then references to dependents are released which makes it possible to garbage collect a stream once any other program-specific reference to it is gone. So the trick is to always unsubscribe with an ofValue when \ we no longer need the stream.

Yes. That is correct.

I think the main problem with the above reasoning is that the library authors are not explicit with their underlying semantics. You've no choice, you'll have to be explicit and formal with your expected behavior (ie have a denotational model). Otherwise you're trying to build your implementation and optimizations on top of something undetermined. So i think that the above approach doesn't make quite sens to me.

You're right that laziness results in semantic problems. The author of Kefir acknowledged that himself. From the very beginning it was very important to me that one can get the value of a stream at any moment someStream(). You've touched on this yourself, this brings us closer to the original semantics of FRP. Here a behavior is semantically a function from time to a value like this Behaviour a :: Time -> a. This is very important and I will not sacrifice it for memory management.

Now, if we choose to not miss any stream event, w'de have to resolve potential memory issues, because streams always hold references to their dependents. we can no longer free a stream by just dereferencing it from our program. So our main issue here is about those ties in the dep. graph that prevents a stream from falling from the tree.

Yes. This happens since Flyd uses a data-driven push implementation. In a demand-driven pull architecture this does not happen. The problem is that the producer (source stream) holds a reference to its consumer (dependent stream) and thus the producer will keep it's consumer alive. In a demand-driven implementation each consumer holds a reference to relevant consumers and thus garbage collection will collect producers when they have not consumers (no references to them). GC favors demand-driven computation. This could be solved with weak references but JavaScript sadly has not such thing.

Unless i miss something, the 2 above rules will enable a stream to be garbage collected by simple deferencement.

Yes. And Flyd already is implemented according to those rules (not only the first). And this works splendidly. But not in the case where a user is creating infinite streams like this s = flyd.stream(1).

A final note about those infinite streams created from DOM elements, i said above that the stream will be freed once the DOM element itself is garbage collected.

How so? If I create a stream like this

var click$ = flyd.stream();
domElement.addEventListener('click', click$);

and a listener is attached to it will never be garbage collected. Not when domElement is removed from the DOM, there will always be a reference click$s dependents to click$. Streams have a reference to their dependencies so that they can detach themselves upon ending (number 2 of your list).

Basically whenever you've created an infinite stream and listeners have been attached to it you will have to end it manually and imperatively by doing someStream.end(true). Otherwise you have a memory leak :(

A little issue however is presented by dynamic collections, if i've a stream representing a dynamic array, all i get from the model is the old and the new array, but no detailed infos. on the change that happened (insertion, removal, reorder or a combination of many of them) so 'ill have to do some ad-hoc synchronisation just as we do in virtual DOM

Indeed. That is handled by flyview. It does children reconciliation somewhat similar to how a virtual DOM library does it.

[yelouafi]

But not in the case where a user is creating infinite streams like this s = flyd.stream(1).

i'm not sure i understand the issue. doesn't a stateful stream like flyd.stream(1), ends immediately because it has no more changes to notify?

Streams have a reference to their dependencies so that they can detach themselves upon ending

You're right, i missed that; there is a mutual observation here, dependents observe change in the sources; and sources observe ends in the dependents.

Basically whenever you've created an infinite stream and listeners have been attached to it you will have to end it manually and imperatively by doing someStream.end(true). Otherwise you have a memory leak :(

i understand here that infinite stream means event streams created without specifying explicit end condition (like the window scroll stream in the Kefir example), and that it doesn't include stateful streams like stream(1). if that so, yes you're right, if a stream depends on an infinite stream it'll be by default an infinite stream unless it specifies another end condition (like stream.take(5)). In this case the necessity of someStream.end(true) will be equivalent to the necessity of the imperative unsubscribe ofValue in Kefir; so unless i miss something again, you get the same feature but without lazy evaluation.

But i think that creating infinite streams without really meaning that is a counter-intuitive idea. If you're going to create an infinite stream at some time t0, then end it imperatively on some later time t1, why not specify this condition directly in a declarative way in the definition of the stream. when it was created

[paldepind]

i'm not sure i understand the issue. doesn't a stateful stream like flyd.stream(1), ends immediately because it has no more changes to notify?

No. It creates a stream that you can push values into:

var foo = flyd.stream(1);
console.log(foo()); // 1
foo(2);
console.log(foo()); // 2

Such a stream can be ended either by doing foo.end(true) or by setting its end stream flyd.endsOn(otherStream, foo). This is not part of FRPs semantics but how Flyd allows one to hook up streams to the world.

[yelouafi]

No. It creates a stream that you can push values into:

Oops! totally forgot this. Indeed, a stream with an imperative push, can only be ended imperatively.

This is not part of FRPs semantics but how Flyd allows one to hook up streams to the world.

I understand, i think this imperative pattern should be handled inside a framework. In flydview for example, instead of the user create himself the streams and hooking them up to the view, the framework itself can be responsible for creating the event streams and delivering them to client components.

the lifecycle of the view can be expressed itself by some other external event, like a user navigating to a different view. and the end of this streams can be itself hooked up onto the end of the containing view.

[rickmed]

@yelouafi @paldepind just wanted to let you know that this was one of the most fascinating reads I've had in quite some time. Bookmarked.