search

TimelyDataflow / timely-dataflow

A modular implementation of timely dataflow in Rust
MIT License
3.22k stars 272 forks source link

Disable asynchronous message transfer? #334

Open cheonruen opened 3 years ago

cheonruen commented 3 years ago

Hi, I am new to Timely Dataflow

I am so impressed by the design philosophy of Timely Dataflow.

Currently, I am investigating the performance impact of its asynchronous execution engine on for some BSP-based algorithms such as PageRank.

Is there any way for me to disable 'asynchronous booster' in the system so that I can check the performance benefit of it?

Thanks!

frankmcsherry commented 3 years ago

Hello.

Thanks for the nice words!

There is no easy way to make message transmission synchronous, without writing your operators to hold back the data themselves. The main point of timely dataflow (and dataflow models generally) is that the movement of data should drive the computation, rather than an external scheduler. So, 100% the system is designed for asynchronous message transfer, to the extent that there isn't a thing for it to be "synchronous" with.

Hope this helps!