Reactive Stream-based programming in the context of MSF (Dunai)

[Rizary]

Hi,

After finishing my google summer of code 2019 using reflex FRP, I am starting to explore more about FRP for the backend (in this case is web API server). I stumbled upon Streamly package to learn more about reactive stream-based server. Following the tutorial I found the claim stated as follows:

The non-determinism, concurrency and streaming combination make streamly a strong FRP capable library as well. FRP is fundamentally stream of events that can be processed concurrently. The example in this tutorial as well as the Streamly.Examples.CirclingSquare example from Yampa demonstrate the basic FRP capability of streamly. In core concepts streamly is strikingly similar to dunai. dunai was designed from a FRP perspective and streamly was originally designed from a concurrency perspective. However, both have similarity at the core.

Based on the above claim, I asked for an explanation from the author about it and got the following explanation:

Vaguely speaking there are two ways to represent problems that involve time domain. One model is FRP and the other one is generally called reactive programming (e.g. http://reactivex.io/). Whereas FRP samples behaviors at different times, reactive programming uses a concurrent task model, generating streams of events and manipulating the streams. Streamly uses the latter model.

The question that I have is:

1. Can we utilize FRP (in this case MSF/Dunai) for stream-based programming on the backend? Is there any current work related to that?
2. Does Dunai itself is enough or should I make another FRP on top of Dunai that specialized in that area?

I am really not sure where to start, but the idea is to have both frontend and backend using FRP. As far as I know, there is already work done in Reflex which use WebSocket to make some frontend widget changed dynamically based on the backend. But I remember that Reflex did not support multithread.

[ivanperez-keera]

That is one extremely well presented issue right there!

Dunai has been extended to support multi-thread FRP, as well as good-old reactive programming.

The first part was work led by @turion, who has created an implementation, has given several talks, and we wrote a paper together (mirror). The idea behind Rhine is that, if two FRP signal functions run concurrently and at different execution rates, then 1) the difference in clocks should be specified at type level, and 2) the connection should be safe to use (and do upsampling or downsampling accordingly).

The second part was mentioned in the original FRP Refactored paper, Sec 6.2, and it's a small implementation of Keera Hails on top of Dunai. Keera Hails is based on a notion of reactive values, which seems very similar to what you mention. They are asynchronous too, and support multi-thread. I would not be surprised if, at least at a theoretical level, we could express Keera Hails as Rhines on an unsafe clock with MVar-like buffers for coordination. Hails is being used for applications, and progressively more in combination with Dunai.

[ivanperez-keera]

For the connection between the Dunai and Keera hails, see also this example: https://github.com/ivanperez-keera/dunai/blob/develop/examples/keerahails/WXText.hs

[ivanperez-keera]

Looking at the type of streamly, I suspect that we could implement their basic StreamD.Type.Stream and StreamK.Type.Stream with some monads, possible based on the Either and the Continuation monads.

If that was the case, then maybe the whole streamly API could potentially be expressed on top of Dunai, and it might 1) bring any benefits of streamly to dunai, 2) simplify streamly, since many constructs should be similar and exist in dunai already.

[Rizary]

Hi @ivanperez-keera

Thank you for your detail explanation. I will look into all the links and start playing with it.

I leave my comment here if there is any further question arise.

[ivanperez-keera]

Sounds great!

[turion]

Ivan has already given a great overview over dunai's connections to concurrency.

About streamly: It has a fundamentally different approach to streaming than dunai. Yes, some of streamly can be reimplemented in dunai because streams are just a special case of stream functions, but streamly's approach circumvents the main point of dunai, which is building safe and synchronous stream functions. Let me explain.

Since Haskell has no productivity checker (like e.g. Agda does), you can write streams that will hang in a loop forever. Let's see. The following function takes two elements of a stream and adds them.

Prelude> sumTwo (a1 : a2 : as) = a1 + a2 : sumTwo as

This function is asynchronous: It eats twice as many elements as it spits out. But that's fine at first:

Prelude> take 10 [1..]
[1,2,3,4,5,6,7,8,9,10]

Now let's use the function recursively:

Prelude> x = 1 : 2 : sumTwo x

That shouldn't work, should it? We can't produce elements fast enough, right? But surprisingly, it does at first:

Prelude> take 3 x
[1,2,3]

Phew. But what if we want more data?

Prelude> take 10 x
[1,2,3^CInterrupted.

It hangs forever after the third element.

Now this example may seem artificial, but this class of bugs is actually a real problem. My point here is that this class of bugs can occur in streamly, but neither in dunai or Rhine. In dunai, it cannot appear by construction because all building blocks are synchronous stream functions. Recursion is done with initialised feedback loops, such as http://hackage.haskell.org/package/dunai-0.5.1/docs/Data-MonadicStreamFunction-Core.html#v:feedback. In Rhine, asynchronicity is guarded by type-level clocks, so the type checker can verify that bugs of this kind cannot occur.

[turion]

Another thing that comes to my mind, but might be due to my ignorance: I don't understand how streamly is an FRP library. It seems to have no notion of time. In dunai, we have bearriver, which is a drop-in replacement for Yampa, and can be implemented on top of dunai with very little code. In Rhine, there is a very rich FRP interface with event clocks and clock-polymorphic behaviours.

But streamly doesn't seem to have that. There is something about rate limitation, but I guess that's more about dealing with load. So if you need to deal with time explicitly, you might be better off with dunai & Rhine.

[Rizary]

@turion very good explanation that will help me understand more about FRP and dunai.

As for streamly, yes I agree with you. The author thinks that it is more of reactivex domain which use concurrent task model to reach the reactivity and not FRP. That is what I don't understand because the claim was it can produce the same goal with FRP in the context of concurrency.

[Rizary]

@ivanperez-keera what is the long term goal of dunai, in my understanding, it is like the umbrella package for FRP library. But I don't see it is used in Rhine as its dependency for example.

So, if I reimplement streamly functionality into dunai, I have to do the same if I want to use Rhine (for clock support) with streamly functionality? or maybe another FRP library which uses similar kind of "monadic stream function" but not use dunai as its dependency.

[ivanperez-keera]

What do you mean? https://github.com/turion/rhine/blob/develop/rhine/rhine.cabal#L102

[Rizary]

I am sorry, I think I misread the Rhine’s package dependencies section in hackage.

Sent from my iPhone

On Oct 23, 2019, at 7:13 AM, Ivan Perez notifications@github.com wrote:

What do you mean? https://github.com/turion/rhine/blob/develop/rhine/rhine.cabal#L102

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[turion]

The best strategy to rebuild streamly from the dunai/rhine world would be reimplementing all synchronous features in dunai and all asynchronous/multi-rate features in rhine.

[Rizary]

@turion thank you for the input!! I will look into both.

[ivanperez-keera]

Since there has been no activity here in over 3 months, I think we can safely close this question.

@Rizary if you manage to reimplement features of streamly on top of dunai, please let us know.