search

pinojs / pino

🌲 super fast, all natural json logger
http://getpino.io
MIT License
14.21k stars 875 forks source link

Support multiple output streams #249

Closed yunong closed 7 years ago

yunong commented 7 years ago

Hi there,

Does Pino support multiple output streams? I took a look at the API and source, and it looks like currently Pino only supports one output stream. We have use cases where we'd like to output the logs both to stdout and HTTP, and multiple output streams, similar to what bunyan has would be required for this use case.

Additionally, if we can change the API to support adding an additional stream after construction, it would also be helpful -- as some streams have to be created asynchronously.

jsumners commented 7 years ago

https://github.com/pinojs/pino-multi-stream provides the requested functionality. We do not implement it in core because it is SLOW. We highly recommend you look at alternate ways of accomplishing the task, e.g. post processing the logs with a transport.

marcbachmann commented 7 years ago

There's also https://www.npmjs.com/package/multi-write-stream if you want the same loglevel for all streams

jsumners commented 7 years ago

@marcbachmann PR for pino-multi-stream would be excellent.

yunong commented 7 years ago

@jsumners thanks for getting back to me so quickly. For multiple streams, can you speak to where the performance penalty comes from?

jsumners commented 7 years ago

@yunong in oder to write a single message to multiple destination streams, each of those destination streams have to be iterated and sent the message.

yunong commented 7 years ago

@jsumners Thanks! That was my assumption as well, but sounds like there are no additional overheads other than the extra stream itself?

jsumners commented 7 years ago

@yunong it entirely depends on what the streams represent. If you look at the benchmark code for pino-multi-stream (https://github.com/pinojs/pino-multi-stream/blob/master/benchmark.js) you'll see that is it merely writing to a bunch of file streams pointing to /dev/null. You can't get any streams that perform much quicker, yet the four stream benchmark results in:

benchBunyanFour*10000: 2654.913ms
benchPinoMSFour*10000: 1282.892ms

That's 1.2, really 1.3, second your application isn't doing what it is designed to do. It is, instead, simply writing log messages.

You should look at https://www.npmjs.com/package/@sematext/logagent and read up on how it can benefit you in combination with Pino -- http://www.nearform.com/nodecrunch/sematext-guest-post-pino-fastest-node-js-logger-production/

jsumners commented 7 years ago

@yunong has your question been answered? Can we close this issue?

davidmarkclements commented 7 years ago

before we do - just to add there's also https://github.com/pinojs/pino-tee by @mcollina

This runs as a separate process (Pino recommends transports as separate processes), so should have faster performance (we should really add benchmarks to pino-tee)

github-actions[bot] commented 2 years ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.