search

TimelyDataflow / differential-dataflow

An implementation of differential dataflow using timely dataflow on Rust.
MIT License
2.58k stars 184 forks source link

Trace imports are invisible #330

Open Kixiron opened 3 years ago

Kixiron commented 3 years ago

No events are emitted that can be used to piece together where an imported trace came from, TraceAgents manage their data internally and .import() uses the source operator to create an entirely new source, so there's no ChannelsEvent linking the two streams nor any sort of differential event that links them.

frankmcsherry commented 3 years ago

Just to make sure that I understand the issue, trace imports are reported (the TraceShare logging event), but not enough breadcrumbs are reported to let you link up the the imported arrangement with the location it was imported to?

Kixiron commented 3 years ago

Yep, precisely. TraceShare just says that an arrangement came from somewhere, but not where it actually comes from

frankmcsherry commented 3 years ago

I think it might be the other way around. TraceAgent records the operator id it was constructed with, and on each importing event increments that. So, each import does announce where it came from, but only where it came from (not, e.g. where it was imported to). Maybe that is what you meant though.

Kixiron commented 3 years ago

Yes, that's my bad