nevalang / neva

🌊 Flow-based programming language with static types and implicit parallelism. Compiles to native code and Go
https://nevalang.org
MIT License
85 stars 7 forks source link

Reduce as higher order component for binary handlers #656

Open emil14 opened 1 month ago

emil14 commented 1 month ago

The Problem

Turns out we've decided to have Map/Filter/For but forgot about Reduce. This comes from:

  1. The fact that reduction is less common operation in programming
  2. We already have some builtin "reducers" like Add

However, user needs to be able to implement custom reducing to solve tasks like "calculate average age of all people in a group".

The need for For/Map/Filter HOCs comes from the fact that it takes an effort in Neva to do iteration without bugs - one need to unpack the stream element, process the data, check if it's last, pack it back if needed and all this should be done with locks to avoid races. HOCs solves these problems by taking care of the race-free predictable iteration. User just passes a handler - a dependency. Just like in FP world.

This is even bigger problem for reduce operation because reduction is stateful, unlike for/map/filter reduce does need to maintain state between iterations (accumulator). That's the nature of reduce.

Proposed Solution

Proposed by @Catya3 in #654 among with any other interesting ideas. The idea is to have generic Reduce higher-order component just like we have Map and For (and will have Filter).

component Main() () {
    nodes { avg Reduce<int>{AvgAge}, Println }
    <stream_of_users> -> avg -> println -> :stop
}

component AvgAge(acc int, cur User) (res int) {
    nodes { Add<int> }
    :acc -> add[0]
    :el -> .age -> add[1]
    add -> :res
}

Please ignore the fact that we Add like it has array-inport instead of accepting a stream, In reality we would need some kind of adapter/convertor. Also let's pretend that we just have a real stream in place of <stream_of_users>

emil14 commented 1 month ago

If Add would be a reducer itself then it would be a bit simpler

component AvgAge(acc int, cur User) (res int) {
    nodes { Add<int> }
    :acc -> add:acc
    :el -> .age -> add:cur
    add -> :res
}

Instead of something like

component AvgAge(acc int, cur User) (res int) {
    nodes { add SomeAdapter{Add<int>} }
    :acc -> add[0]
    :el -> .age -> add[1]
    add -> :res
}