This adds a batcher API to help with async/multi-frame flows. The batcher can gather data from one or more sets of agents; and delay actual execution until a later point. This helps achieve larger batch sizes, which leads to better amortized performance as small batches degenerate into Matrix-Vector multiplications.
More details in the changelog file; which is a good starting point for review.
Graph for performance. Orange = new. Lower = better.
Description of Changes
This adds a batcher API to help with async/multi-frame flows. The batcher can gather data from one or more sets of agents; and delay actual execution until a later point. This helps achieve larger batch sizes, which leads to better amortized performance as small batches degenerate into Matrix-Vector multiplications.
More details in the changelog file; which is a good starting point for review.
Graph for performance. Orange = new. Lower = better.
Fixes #24, #8