nevalang / neva

🌊 Flow-based programming language with static types and implicit parallelism. Compiles to native code and Go
https://nevalang.org
MIT License
85 stars 7 forks source link

`$const` can fill the buffer real quick #681

Open emil14 opened 1 week ago

emil14 commented 1 week ago

New component sends messages in a infinite loop and that could create lots of useless events and also fill the buffer real quick. The bigger buffer is the more memory we will waist. With big buffers and big programs (lots of consts) we can run of out memory


Related to #677 and #671

emil14 commented 1 week ago

Restrict to deferred connections

Allow to use constants only in conjunction with Lock or -> ()

Problem

Deferred connection doesn't mean that sending of a constant is deferred, it means that receiving is deferred

It means we can fill the Lock's buffer

Other

emil14 commented 1 week ago

Inport for New

Main (but not the only) reason for deferred connections is sending constants. They are sent by using New component that sends messages in an infinite loop.

There's a reason for that - let's say there's a and b inports. Let a be real dynamic data and b "static" (connected to const). How do we reactivate when new a arrive? Infinite loop solves that problem.

However, this lead to problem described in this issue. If we add buffer, then constants will fill that buffer real quick. Infinite loop works super fast. But what if we add inport to New? After all, that was the initial reason for -> (), isn't it?

Before

:start -> (42 -> println)

After

:start -> 42 -> println

No need for deferred connection in such a simple case. Also that opens a door for multiple senders:

[:foo, :bar] -> 42 -> println

But it was always possible isn't it?

[:foo, :bar] -> (42 -> println)

Problem - more complexity in networks

At least one problem is that there are other cases where we don't need deferred connections now.

We do need them (speaking about $const) when constant is the only activation condition (like that one simple example). But otherwise we might not need it.

For instance this example is completely fine:

:a -> add[0]
42 -> add[1]

Even though 42 is always "there" add will wait for dynamic data from :a. We could rewrite this example by using deferred connections without changing it's semantic behaviour (but adding extra connections to desugared version):

:a -> [
  add[0],
  (42 -> add[1])
]

I would say it looks much worse. It's possible to imagine complex programs where this makes a huge difference. However, what we are talking about here is a little different thing. Let's add inport to New and see how it changes our program

:a -> [
  add[0],
  42 -> add[1]
]

I would say it look a little better than with deferred connections version but still much worse than original version

emil14 commented 1 week ago

Do not use real connections and transmission for constants

From performance point of view that would be perfect.

We don't really have to send constants across the network. We could kinda "remember" them inside inport object. So every time runtime function calls Receive() function it just gets the same value. No channels - no buffers and no buffers means no problems! As a bonus we do not waste CPU cycles on transmission of constants between ports.

msg := inport.Receive() // msg == 42 always

...

func (inport) Receive() Message {
  return IntMessage(42)
}

Problem

Combination with normal values

:sig -> ($a -> foo)
bar -> foo

Here we send const $a to foo node each time we got msg from sig but also each time bar node sends a message.

It's unclear how to handle this with proposed design where foo's inport must have one bound value $a

Is it possible to restrict usage of constants and forbid combination? What FBP says about that with its IIPs?