Previously, we created a connection once and expected the StatsdClient to hold
it until closed. For long-lived processes running in an environment where the
DNS name of the statsd server might resolve to different IP addresses over
time, this behavior is undesirable. When the IP address of the server changes,
the client begins to silently fail to send metrics (as its client holds a
connection pointing at the old server).
Instead, we simply create a new socket for each metric sent. While this is
natural for the connectionless UDP, it does impose some runtime overhead and
moves the risk of a timeout (for UDP, primarily around name resolution) from
initialization to later points in a process' runtime. This risk is not
addressed here, but could be mitigated by:
making the timeout on creation more aggressive (currently 1 second)
adding explicit caching of the connection, such that it is only
re-created periodically
It would also be possible to make the interface slightly nicer by
automatically creating the connection; that is not addressed here, either.
Previously, we created a connection once and expected the StatsdClient to hold it until closed. For long-lived processes running in an environment where the DNS name of the statsd server might resolve to different IP addresses over time, this behavior is undesirable. When the IP address of the server changes, the client begins to silently fail to send metrics (as its client holds a connection pointing at the old server).
Instead, we simply create a new socket for each metric sent. While this is natural for the connectionless UDP, it does impose some runtime overhead and moves the risk of a timeout (for UDP, primarily around name resolution) from initialization to later points in a process' runtime. This risk is not addressed here, but could be mitigated by:
It would also be possible to make the interface slightly nicer by automatically creating the connection; that is not addressed here, either.