TimelyDataflow / timely-dataflow

A modular implementation of timely dataflow in Rust
MIT License
3.29k stars 272 forks source link

builder_ref: first step towards ref based operator builder #440

Open petrosagg opened 2 years ago

petrosagg commented 2 years ago

This commit is the first step towards building an operator builder that doesn't rely on holding on to Rc<RefCell<ChangeBatch<T>>.

In order to do so builder_ref::OperatorBuilder encodes the current shape of the operator in the type system by consuming self and producing a new builder type every time an input or output is added.

When the time comes to build the operator, the system requires the user to supply a closure whose type signature matches exactly the inputs and outputs have have been previously declared.

This PR only adds the typestate machinery to builder_ref::OperatorBuilder. A subsequent PR will work on removing all the shared refcounted change batches since they now have a single owner. The reason this isn't included in this PR is to keep it at a reasonable size since many auxiliary types like the handles will need to be adjusted to work with references.