Using UDS (Unix Domain Sockets) is one of the main recommendation from Datadog to achieve higher throughput for StatsD traffic. So far, this client did not provide such a sink.
In this PR I am introducing a UDS sink that uses same batching logic from UDP sink.
Follow up improvements
https://github.com/Shopify/statsd-instrument/pull/374 - Here I am removing the specific types of sinks and making them generic, then I introduce the concept of Connection which will then interact with the sockets. This solves the duplication problem I introduced in this PR.
Ended up merging this PR above and squashing the other commits, when reviewing, one can review the commits separately.
Benchmarks
We can see from benchmarks that using larger batches improves client throughput with a considerable margin:
▶ RUBY_YJIT_ENABLE=true ./benchmark/local-udp-throughput
Logs are stored in /var/folders/9v/vqq8s1cd2n53w68813dwjz4c0000gn/T/statsd-instrument-benchmarks
===== UDP sync throughput (5 threads) - total events: 150000 =====
events: 138,061.4/s
===== UDP batched throughput (5 threads) - total events: 150000 =====
events: 734,325.8/s
===== UDS batched with small packet throughput (5 threads) - total events: 150000 =====
events: 1,386,411.3/s
===== UDS batched with jumbo packet throughput (5 threads) - total events: 150000 =====
events: 1,248,824.0/s
Summary
Using UDS (Unix Domain Sockets) is one of the main recommendation from Datadog to achieve higher throughput for StatsD traffic. So far, this client did not provide such a sink.
In this PR I am introducing a UDS sink that uses same batching logic from UDP sink.
Follow up improvements
Connectionwhich will then interact with the sockets. This solves the duplication problem I introduced in this PR.Ended up merging this PR above and squashing the other commits, when reviewing, one can review the commits separately.
Benchmarks
We can see from benchmarks that using larger batches improves client throughput with a considerable margin: