[X] Replace custom :ets-based worker pool with worker_pool
Note: I'm not really happy with the solution I landed on for creating multiple concurrent worker pools.
Work remaining:
[ ] Implement packet buffering, with configurable number of bytes (e.g. for UDP; base it off of MTU minus the statsd-related header. For UDS; 8192 bytes is recommended. See here for a good guide about tuning for high-throughput metrics)
Currently, the tests use a assert_reported/2 helper function which calls :gen_udp.recv. This will likely have to change to accommodate larger packets containing many metrics.
[ ] Document packet buffering
[ ] Expose choice of worker selection (I think hash_worker will work well here)
In particular, with hash_worker, it becomes possible to do client-side aggregation of metrics. For example, if the foo.bar counter is incremented once 10,000 times, all those increments go to the same worker, which could emit a single line in an outgoing packet for all 10,000 increments. This type of optimization is less effective with different choices of worker selection.
Work done so far:
UDP
from a struct to a GenServerWork remaining:
assert_reported/2
helper function which calls:gen_udp.recv
. This will likely have to change to accommodate larger packets containing many metrics.hash_worker
, it becomes possible to do client-side aggregation of metrics. For example, if thefoo.bar
counter is incremented once 10,000 times, all those increments go to the same worker, which could emit a single line in an outgoing packet for all 10,000 increments. This type of optimization is less effective with different choices of worker selection.