gliderlabs / logspout

Log routing for Docker container logs
MIT License
4.64k stars 679 forks source link

logspout is not sending logs after some hours just reconnecting multiple time #386

Open Deepak1100 opened 6 years ago

Deepak1100 commented 6 years ago

I was not receiving logs from some of my machines and the time at which they stop sending logs are different. sometimes it sends the log of some container on the same machines while the logs of the other container on that very same machine are not getting sent. these are the environment variable for my logspout container.

 "BACKLOG=false",
 "EXCLUDE_LABEL=logspout.exclude",
 "INACTIVITY_TIMEOUT=30s",
 "SYSLOG_TAG={{.Container.Config.Labels.source}}",

I have restarted my logspout container but it still giving the same error. unless i remove my complete ec2 instance and bring new one with logspout and other app it don't work. below is the log of logspout.

2018/03/22 10:47:00 syslog: reconnect successful
2018/03/22 10:47:00 syslog: write tcp 172.17.0.2:50086->10.25.20.210:6010: write: connection reset by peer
2018/03/22 10:47:00 syslog: reconnecting up to 10 times
2018/03/22 10:47:00 syslog: reconnect successful
2018/03/22 10:47:00 syslog: write tcp 172.17.0.2:50088->10.25.20.210:6010: write: connection reset by peer
2018/03/22 10:47:00 syslog: reconnecting up to 10 times
2018/03/22 10:47:00 syslog: reconnect successful
2018/03/22 10:47:00 syslog: write tcp 172.17.0.2:50090->10.25.20.210:6010: write: connection reset by peer
2018/03/22 10:47:00 syslog: reconnecting up to 10 times
2018/03/22 10:47:00 syslog: reconnect successful
2018/03/22 10:47:00 syslog: write tcp 172.17.0.2:50092->10.25.20.210:6010: write: connection reset by peer
2018/03/22 10:47:00 syslog: reconnecting up to 10 times
2018/03/22 10:47:00 syslog: reconnect successful
2018/03/22 10:47:00 syslog: write tcp 172.17.0.2:50094->10.25.20.210:6010: write: connection reset by peer
2018/03/22 10:47:00 syslog: reconnecting up to 10 times
2018/03/22 10:47:00 syslog: reconnect successful
2018/03/22 10:47:00 syslog: write tcp 172.17.0.2:50096->10.25.20.210:6010: write: connection reset by peer
2018/03/22 10:47:00 syslog: reconnecting up to 10 times
2018/03/22 10:47:00 syslog: reconnect successful
2018/03/22 10:47:00 syslog: write tcp 172.17.0.2:50098->10.25.20.210:6010: write: connection reset by peer
2018/03/22 10:47:00 syslog: reconnecting up to 10 times
2018/03/22 10:47:00 syslog: reconnect successful
2018/03/22 10:47:00 syslog: write tcp 172.17.0.2:50100->10.25.20.210:6010: write: connection reset by peer
2018/03/22 10:47:00 syslog: reconnecting up to 10 times
2018/03/22 10:47:00 syslog: reconnect successful
2018/03/22 10:47:00 syslog: write tcp 172.17.0.2:50102->10.25.20.210:6010: write: connection reset by peer
2018/03/22 10:47:00 syslog: reconnecting up to 10 times
2018/03/22 10:47:00 syslog: reconnect successful
2018/03/22 10:47:00 syslog: write tcp 172.17.0.2:50104->10.25.20.210:6010: write: connection reset by peer
2018/03/22 10:47:00 syslog: reconnecting up to 10 times
2018/03/22 10:47:00 syslog: reconnect successful
2018/03/22 10:47:00 syslog: write tcp 172.17.0.2:50106->10.25.20.210:6010: write: connection reset by peer
2018/03/22 10:47:00 syslog: reconnecting up to 10 times
2018/03/22 10:47:00 syslog: reconnect successful
2018/03/22 10:47:40 syslog: write tcp 172.17.0.2:50108->10.25.20.210:6010: write: connection reset by peer
2018/03/22 10:47:40 syslog: reconnecting up to 10 times
2018/03/22 10:47:40 syslog: reconnect successful
2018/03/22 10:48:00 syslog: write tcp 172.17.0.2:50136->10.25.20.210:6010: write: connection reset by peer
2018/03/22 10:48:00 syslog: reconnecting up to 10 times
2018/03/22 10:48:00 syslog: reconnect successful
2018/03/22 10:48:00 syslog: write tcp 172.17.0.2:50156->10.25.20.210:6010: write: connection reset by peer
2018/03/22 10:48:00 syslog: reconnecting up to 10 times
2018/03/22 10:48:00 syslog: reconnect successful
2018/03/22 10:48:00 syslog: write tcp 172.17.0.2:50158->10.25.20.210:6010: write: connection reset by peer
2018/03/22 10:48:00 syslog: reconnecting up to 10 times
2018/03/22 10:48:00 syslog: reconnect successful
2018/03/22 10:48:00 syslog: write tcp 172.17.0.2:50164->10.25.20.210:6010: write: connection reset by peer
2018/03/22 10:48:00 syslog: reconnecting up to 10 times
2018/03/22 10:48:00 syslog: reconnect successful
2018/03/22 10:48:00 syslog: write tcp 172.17.0.2:50166->10.25.20.210:6010: write: connection reset by peer
2018/03/22 10:48:00 syslog: reconnecting up to 10 times
2018/03/22 10:48:00 syslog: reconnect successful
2018/03/22 10:48:00 syslog: write tcp 172.17.0.2:50168->10.25.20.210:6010: write: connection reset by peer
2018/03/22 10:48:00 syslog: reconnecting up to 10 times
2018/03/22 10:48:00 syslog: reconnect successful
2018/03/22 10:48:00 syslog: write tcp 172.17.0.2:50172->10.25.20.210:6010: write: connection reset by peer
2018/03/22 10:48:00 syslog: reconnecting up to 10 times
2018/03/22 10:48:00 syslog: reconnect successful
2018/03/22 10:48:00 syslog: write tcp 172.17.0.2:50174->10.25.20.210:6010: write: broken pipe
2018/03/22 10:48:00 syslog: reconnecting up to 10 times
2018/03/22 10:48:00 syslog: reconnect successful
2018/03/22 10:48:00 syslog: write tcp 172.17.0.2:50176->10.25.20.210:6010: write: connection reset by peer
2018/03/22 10:48:00 syslog: reconnecting up to 10 times
2018/03/22 10:48:00 syslog: reconnect successful
2018/03/22 10:48:00 syslog: write tcp 172.17.0.2:50178->10.25.20.210:6010: write: connection reset by peer

i have tried restarting syslog as well and there isn't any error over there.

forestjohnsonpeoplenet commented 4 years ago

https://github.com/gliderlabs/logspout/issues/377#issuecomment-537708428