fluent / fluent-logger-node

A structured logger for Fluentd (Node.js)
Apache License 2.0
259 stars 83 forks source link

Failed to call _flushSendQueue() during frequent log emit #168

Open hlhl040 opened 4 years ago

hlhl040 commented 4 years ago

Hi, We're using fluent-logger-node v3.4.1 to send logs from Lambda functions. During our test, we emit 300KB logs every 100ms for 60 seconds, so around 180MB logs in total. During the test, the first emit() started a socket connection(see code below) if (this.tls) { this._socket = tls.connect(Object.assign({}, this.tlsOptions, { host: this.host, port: this.port }), () => { postConnect(); }); addHandlers(); } Somehow the call back function was not called until the last emit. For the following emit(), the _connect() returned immediately because it thought still in a connecting process. So all logs were pushed into the queue and flushed once after the last emit(). I wonder if this is the expected behavior because we may run out of memory if log size is big.

okkez commented 4 years ago

Could you show me reproducible code and steps without Lambda?