Recommended setup for production

david-szabo97 commented 2 years ago

The documentation contains a mixed use of pino.transport, pino.destination, pino.multistream, and Pino created without arguments. Legacy vs Pino v7+ transports is also confusing.

What's the recommended setup for production? Production = least overhead, best performance, logging to stdout

My first guess would be

const logger = pino(pino.destination());

After reading the documentation, I've found a paragraph:

The difference between using the pino/file transport builtin and using pino.destination is that pino.destination runs in the main thread, whereas pino/file sets up pino.destination in a worker thread.

This makes pino.destination a wrong choice because it runs in the main thread. Therefore to get the best performance out of Pino we must use v7+ transports.

In our case we want to log to stdout, therefore we create a pino/file transport:

const transport = pino.transport({
  target: "pino/file",
});

const logger = pino(transport);

Having a section in the documentation about running Pino in production would be great. Unfortunately, there are too many options at the moment, which can confuse new users of Pino (or even existing users like myself).

mcollina commented 2 years ago

The default settings are defined to work well in your simple production setup. You don't need to tweak anything.

All the above are required if you have more sophisticated requirements.

Would you like to send a PR to clarify this further in our docs?

david-szabo97 commented 2 years ago

By default settings you mean this, right?

const logger = pino();

According to the documentation, the default destination is pino.destination(1). So the above is equivalent to the following:

const logger = pino(pino.destination(1));

As far as I understand, pino.destination runs on the main thread in this case. So the following should perform better because the logging is delegated to a worker thread:

const transport = pino.transport({
  target: "pino/file",
});

const logger = pino(transport);

Benchmark - based on the `basic-bench.js` - seems to agree with my assumptions:

benchPino*10000: 372.211ms
benchPinoMinLength*10000: 132.718ms
benchPinoNodeStream*10000: 417.469ms
benchPDestinationDevNull*10000: 346.205ms
benchPfile*10000: 77.384ms

benchPDestinationDevNull = pino(pino.destination("/dev/null")) benchPfile = pino(pino.transport({ target: "pino/file", options: { destination: "/dev/null" } })) Rest is the same as in basic-bench.js, all goes to /dev/null

/dev/null is messing up the benchmark?

Instead of sending the logs to /dev/null, I sent them to process.stdout. I've also redirected the output to a file.

Command: node bench.js > bench.log

Loggers: benchPDestination = pino() benchPfile = pino(pino.transport({ target: "pino/file" }))

Results

benchPDestination*10000: 82.929ms
benchPfile*10000: 79.111ms

Looks like both take the same amount of time when printing to stdout. Ran the benchmark multiple times and it's head-to-head all the time. Sometimes one wins with 10%, sometimes the other win with 10%. Therefore we can take this as a draw.

Question

Are there any benefits of using the pino/file transport over using pino.destination when logging to the stdout stream? (basically worker thread VS main thread?)

Would it make sense to recommend pino/file over pino.destination in production?

IMO, pino/file should be a better choice in production because the logging happens in the worker thread entirely. On the other hand, it might be unnecessary because the destination is stdout.

EDIT: Yes, I'm overthinking this 😄 I'm not worrying about performance, I'm driven by curiosity.

mcollina commented 2 years ago

Are there any benefits of using the pino/file transport over using pino.destination when logging to the stdout stream? (basically worker thread VS main thread?)

No. This just adds more overhead due to moving the data off the main thread. However this is available for people that want to log to multiple destinations, e.g. to elastic search and to a file. In that case the file transport is useful.

Would it make sense to recommend pino/file over pino.destination in production?

No.

Yes, I'm overthinking this 😄 I'm not worrying about performance, I'm driven by curiosity.

Indeed! But curiosity is great!

janeklb commented 1 year ago

No. This just adds more overhead due to moving the data off the main thread. However this is available for people that want to log to multiple destinations, e.g. to elastic search and to a file. In that case the file transport is useful.

I was looking for this exact bit of information. From reading the documentation, it was not clear that the default mode (const logger = require('pino')()) yields the best performance for basic / stdout-only logging. I re-read the transports docs a few times thinking that I must have missed something.

Having said all that...

Would you like to send a PR to clarify this further in our docs?

I tried to think about how to fit this into the docs as they stand, and was not able to come up with a good solution. Maybe, simply, an entry to help?

pinojs / pino