search

cloudfoundry-community / firehose-to-syslog

Send firehose events from Cloud Foundry to syslog.
MIT License
44 stars 58 forks source link

Diode Set Collision #179

Closed bradgoodman closed 6 years ago

bradgoodman commented 6 years ago

We just pushed the new nozzle out today and received this error. Any ideas?

   2018-02-26T17:10:25.41-0500 [APP/PROC/WEB/1] ERR 2018/02/26 22:10:25 Diode set collision: consider using a larger diode
   2018-02-26T17:10:28.11-0500 [APP/PROC/WEB/1] ERR [2018-02-26 22:10:28.113798814 +0000 UTC m=+97.337435301] Exception occurred! Message: Missed Logs  Details: 10000
   2018-02-26T17:11:14.50-0500 [APP/PROC/WEB/1] ERR 2018/02/26 22:11:14 Unsolicited response received on idle HTTP channel starting with "HTTP/1.0 408 Request Time-out\r\nCache-Control: no-cache\r\nConnection: close\r\nContent-Type: text/html\r\n\r\n<html><body><h1>408 Request Time-out</h1>\nYour browser didn't send a complete request in time.\n</body></html>\n"; err=<nil>
shinji62 commented 6 years ago

Yes , buffer is full. So you need to increase buffer or need to consume logs faster.

Prefer the later one.

Thanks Envoyé de mon iPhone

Le 27 févr. 2018 à 07:22, bradgoodman notifications@github.com a écrit :

We just pushed the new nozzle out today and received this error. Any ideas?

2018-02-26T17:10:25.41-0500 [APP/PROC/WEB/1] ERR 2018/02/26 22:10:25 Diode set collision: consider using a larger diode 2018-02-26T17:10:28.11-0500 [APP/PROC/WEB/1] ERR [2018-02-26 22:10:28.113798814 +0000 UTC m=+97.337435301] Exception occurred! Message: Missed Logs Details: 10000 2018-02-26T17:11:14.50-0500 [APP/PROC/WEB/1] ERR 2018/02/26 22:11:14 Unsolicited response received on idle HTTP channel starting with "HTTP/1.0 408 Request Time-out\r\nCache-Control: no-cache\r\nConnection: close\r\nContent-Type: text/html\r\n\r\n

408 Request Time-out

\nYour browser didn't send a complete request in time.\n\n"; err= — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

bradgoodman commented 6 years ago

Would you suggest just adding additional instances to handle the load?

shinji62 commented 6 years ago

Well firehose-to-syslog, do not really consume logs is just a bridge (with some logic) between your syslog endpoint and the CF firehose.

Adding more instance will be the same as adding a bigger buffer (+pushing with more memory). This not really resolve your issue, maybe you need to check why your syslog endpoint is slow to ingest logs.