mcollina
commented
3 days ago There are no performance benefits in using transports to write to a file. The main benefit is to allow external transports (Elastic, Loki, syslog, etc) to be able to flush data before the main process exits, preventing the loss of data. This also increase performance by reducing the memory load in the main thread, and allowing better batching. However, pino.destination() provides peek performance to write things to a file.
I think you should be able to write to the same file from different threads, as long as you write to the same file descriptor. I have never seen an issue with stdout, but that might be special.
bmacnaughton
We use pino and are quite happy with the its performance and function. I recently spent some time walking through it and have some questions to 1) check my understanding of what's going on and 2) understand the benefit of spawning multiple worker threads that write to a single file. I understand that point 2 might be an uncommon case; we encounter it because we invoke pino in both node's ESM loader thread and in the main thread. Both write to the same log file.
From my reading:
So the only work that seems to be done in the worker thread is to write to the stream (and the Atomics synchronization).
Considering that the worker thread then relies on libuv to handle the actual disk IO, it seems like the worker thread is another layer that introduces some latency, memory, synchronization, scheduling, etc.
The benefit I see is that it allows multiple threads to write to the same file transparently without having any central controller.
My main question is that the docs suggests it's about performance, but my (admittedly less than perfect) understanding of how it works looks like it's more about being able to write to the same file from multiple threads.
I appreciate any clarity you can provide.