search

fluent / fluent-logger-node

A structured logger for Fluentd (Node.js)
Apache License 2.0
259 stars 83 forks source link

Too many open sockets? #78

Closed chrisrecalis closed 7 years ago

chrisrecalis commented 7 years ago

We just recently upgraded 2.4.2 as of yesterday and immediately noticed severe network issues in our cluster. Just to give a little background on our deployment. We are running a kubernetes cluster where fluentd is deployed as deamon-set. This allows any app in the cluster to connect and push data locally without having to jump the network.

I just quickly went through #77 which was just merged in and published to npm. If I'm not understanding the code correctly let me know but It looks like a new socket is being created on every single flush. If we are doing 500~1000 emits to the logger per second this is a super high number of sockets being created. Which affects both the app and any other applications running on the same vm.

https://github.com/fluent/fluent-logger-node/blob/8da51327300d2fbffa09dd5cf51ced5fb152085e/lib/sender.js#L178-L209

okkez commented 7 years ago

Sorry for my mistake. I've confirmed fd leak with v2.4.2. I've reverted #77 and release new version v2.4.3. I've tested v2.4.3 and I've confirmed no fd leak with it.

Thank you for reporting issue quickly!

okkez commented 7 years ago

@chrisrecalis Could you close this issue when your problem is fixed by v2.4.3?