Closed Houlistonm closed 2 years ago
We upgraded from td-agent-3.6.0 to td-agent-3.8.0 and our Grafana/Prometheus was reporting NO DATA for the fluent pipeline. Our td-agent,log had this message in it.
2020-07-03 14:50:47 +0000 [error]: #0 #<Async::Task:0x3fd56383151c incoming connection #<Addrinfo: 127.0.0.1:39414 TCP> [fd=153] (failed)>Broken pipe
There were no other messages to indicate the source process/task to we started reviewing the change logs and recent issues with the embedded gems.
Found this issue : https://github.com/fluent/fluent-plugin-prometheus/issues/159
We hand patched GEM with the fix in this PR: https://github.com/fluent/fluent-plugin-prometheus/pull/161
And our metrics began working and the broken pipe errors have not returned.
We upgraded from td-agent-3.6.0 to td-agent-3.8.0 and our Grafana/Prometheus was reporting NO DATA for the fluent pipeline. Our td-agent,log had this message in it.
2020-07-03 14:50:47 +0000 [error]: #0 #<Async::Task:0x3fd56383151c incoming connection #<Addrinfo: 127.0.0.1:39414 TCP> [fd=153] (failed)>Broken pipeThere were no other messages to indicate the source process/task to we started reviewing the change logs and recent issues with the embedded gems.
Found this issue : https://github.com/fluent/fluent-plugin-prometheus/issues/159
We hand patched GEM with the fix in this PR: https://github.com/fluent/fluent-plugin-prometheus/pull/161
And our metrics began working and the broken pipe errors have not returned.