A "writer" is a component that accepts datapoints/spans from a channel
and deal with buffering/sending them in a manner that provides some
trade-off between prioritization and concurrency/batching of
requests (i.e. latency) to ingest.
The first implementation uses a ring/circular buffer to minimize
allocations and prioritize newer datapoints over older ones when
dropping occurs. It uses the common pattern of batching as much as
possible until the input channel runs dry (or a max batch size is hit).
This has been in use in the Smart Agent for many months now without
issue. It was largely copied from there with many enhancements.
We have implemented this same thing many times so it seems good to put
it in a common place where it is easily reused.
Rework of https://github.com/signalfx/golib/pull/146 and putting it here in this repo.
A "writer" is a component that accepts datapoints/spans from a channel and deal with buffering/sending them in a manner that provides some trade-off between prioritization and concurrency/batching of requests (i.e. latency) to ingest.
The first implementation uses a ring/circular buffer to minimize allocations and prioritize newer datapoints over older ones when dropping occurs. It uses the common pattern of batching as much as possible until the input channel runs dry (or a max batch size is hit). This has been in use in the Smart Agent for many months now without issue. It was largely copied from there with many enhancements.
We have implemented this same thing many times so it seems good to put it in a common place where it is easily reused.