TimelyDataflow / differential-dataflow

An implementation of differential dataflow using timely dataflow on Rust.
MIT License
2.54k stars 183 forks source link

Downgrade/upgrade trace agents #401

Closed antiguru closed 8 months ago

antiguru commented 1 year ago

Add the ability to downgrade trace agents to only hold weak references to the trace. The weak reference can be upgraded and dereferenced to the trace, but not upgraded to a full agent. This is because the caller already gave up the right to prevent compaction, and re-acquiring it, while technically possible, introduces uncertainty about what time can be selected.

Example

let weak_trace = arranged.trace.downgrade();

if let Some(trace) = weak_trace.upgrade() {
    trace.as_deref().map_batches(|_| {});
}
antiguru commented 8 months ago

We implemented an alternative in #404.