Closed pv4 closed 1 year ago
Hmm... this issue seems to be a copy of #120. But this one has a patch. Besides, the solution suggested in #120 may not work properly sometimes if connection or grafana are slow or if batching is off.
+1 We are also having this issue where the node process ends before the log is sent to graphana 😢
Thanks for the patch, and sorry for the delayed response. :) Released this patch under the @next
tag, version 6.0.7-rc2
.
Hello!
I use winston-loki in many projects and in case of unfixable runtime errors I log the error and throw an uncaughtable exception to end nodejs execution. Something like the following (I replaced the
throw
byprocess.exit(0)
to simplify the output if you run the demo yourself):But in this case the error doesn't get to grafana even with
batching: false
. It appears to be related to asynchronous nature of nodejs. Briefly, logger.log() doesn't mean "send the log to grafana" but instead it means "start sending the log to grafana and return execution to the caller without waiting the response from grafana". In the case above right after the startprocess.exit(0)
is called and nodejs exits execution without waiting until the logs are really received by grafana. Therefore the logs are highly unlikely to get to grafana leaving me without a knowledge about what exactly happened.Currently I have to do something ugly like the following to ensure the logs appear in grafana:
But that's ugly and may not work if the network is slow and/or grafana requires more than 5000 milliseconds to get and process the logs.
A better fix for this issue is in the patch attached. After applying the patch to winston-loki today's development HEAD the code can be changed to the following to get the desired behavior:
The line
await loggerTransport.flush();
does exactly what is written in jsdoc in the patch. It waits until all logs "queued" before (by callinglogger.log()
or similar) are sent to grafana, the responses are received and processed by winston-loki, and only then continues execution. By the time ofprocess.exit(0)
call grafana has RESPONDED winston-loki that all the logs are received and stored.After investigating winston-loki sources I would say it's the best solution technically possible. Tha patch works correctly with batching both turned on and turned off.
Could you please consider applying the patch upstream and making a release including it?
winston-loki-flush.patch.gz