search

cloudfoundry-community / logsearch-for-cloudfoundry

A Logsearch addon that customises Logsearch to work with Cloud Foundry data
Apache License 2.0
49 stars 58 forks source link

Logstash ingestors loses connection to firehose #307

Closed mordax7 closed 5 years ago

mordax7 commented 5 years ago

ingestor_syslog (connection between firehose and elasticsearch and home to the filterchain) In regular intervals, those ingestors stop working. To get them going again, one has to stop the ingestor_syslog and ingestor_cloudfoundry-firhose process and start them again in reverse order. There is no apparent reason given in the logs of either of those 3 services. The only hint we were able to find, was that the ingestor_syslog seems not to accept connections anymore Failed to fire hook: dial tcp 127.0.0.1:5514: getsockopt: connection refused (taken from the log of the ingestor_cloudfoundry-firehose). The ingestor_syslog process listening on this port, however, is up and running it seems.

My belief is that we lose the connection because the massive amounts of logs can cause problems for the client who streams these logs via websocket.

Anyone had a similar problem?

mordax7 commented 5 years ago

Got fixed with newest release v210.0.0