search

cda-group / arcon

State-first Streaming Applications in Rust
https://cda-group.github.io/arcon/
Apache License 2.0
175 stars 17 forks source link

chore(refactor): ChannelStrategy/OperatorContext rework #250

Closed Max-Meldrum closed 3 years ago

Max-Meldrum commented 3 years ago

Closes #248 Closes #119

  1. Reduce the number of generic types required for OperatorContext
  2. Remove a bunch of unsafe calls inside Node as Operator no longer needs a Cell
  3. ChannelStrategy now returns (Channel, ArconMessage) if there is something to send...
Max-Meldrum commented 3 years ago

The last commit changes the Operator/OperatorContext quite a bit.

  1. State of an Operator now lives within a Node and can be accessed through the OperatorContext ctx.state()
  2. Node now holds 1 OperatorContext that is borrowed out when calling operator methods.
  3. OperatorBuilder now requires a state constructor.
  4. Introduce Window Index (Appender, Incremental, Arrow)
  5. Redesign WindowAssigner to use WindowState struct that has assigner state + a user-defined Window Index
  6. Pass OperatorBuilder struct to the NodeManager so that it can construct Operators / States during runtime.
  7. Change empty ArconState from () to the EmptyState struct.
  8. Arcon timer is passed to the OperatorContext as Box<dyn Timer<...>> with backend type erased.