Closed awwit closed 3 years ago
@Eomm could you take a look?
@mcollina @Eomm I looked in more detail what could be the problem and this is what I found out:
Transferring data from pino-syslog
further down the pipeline is only possible if I change the code here: https://github.com/pinojs/pino-abstract-transport/blob/main/index.js#L11-L44
From:
const steam = split(function (line) { ... })
To:
const stream = new PassThrough()
But then there is no data transformation.
It looks like the split
stream is receiving no input. (From pino-sislog
to my transport.)
I still don't understand why this is happening.
@mcollina @Eomm hi guys!
So, I figured out what was the matter.
The problem is really in the pino-abstract-transport
module. Internally, you use the split2
module, which passes data on only if it encounters line breaks.
The pino-syslog
module works well and does everything right. It converts the logs to syslog format and then the data goes to the split2
stream. In which the data gets stuck, due to the fact that a line break is expected.
In general, it seemed to me inappropriate to use the split2
module inside the pino-abstract-transport
. It turns out that each transport must transfer data with line breaks, otherwise it will not work.
It would be better to move this functionality (splitting data into lines) into a separate transport, which is connected to the pipeline like all other plugins.
This explains why the tests were working
...getConfigPath('5424', 'newline.json')
I just pick one random file configuration
So this is a workaround actually
Unfortunately, the baseline syslog format (https://datatracker.ietf.org/doc/html/rfc3164#section-4.1.3) states:
The MSG part will fill the remainder of the syslog packet. This will usually contain some additional information of the process that generated the message, and then the text of the message. There is no ending delimiter to this part.
Also, modern syslog also does not specify an "ending delimiter" -- https://datatracker.ietf.org/doc/html/rfc5424#section-6
pino-syslog
was only ever intended to receive NDJSON and format each individual line into a syslog formatted line.
Here is an issue for pino-socket: https://github.com/pinojs/pino-socket/issues/46.
@mcollina @Eomm I apologize, I made a mistake here myself when creating an issue.
There are no problems in any of your modules (pino-syslog
and pino-socket
). The pino-socket
module does not use pino-abstract-transport
at all, which means it has no problems with split2
.
The problem just arose with my own transport when I implemented it via pino-abstract-transport
. This only means that it is impossible to add two modules to the pipeline implemented through pino-abstract-transport
.
I apologize to you guys for misleading you.
Hi!
I made such a configuration as you described in the example:
But I don't get any logs.
I spent a lot of time debugging this issue. And here's what I found out:
Your plugin (pipo-syslog) does not transmit any data further down the pipeline. To be sure of this, I wrote my transport and replaced pino-socket:
After starting the logger, I only see the message of
Console-1
. When I try to send data to the logger, nothing happens. I don't see a single message.But if I comment out your plugin in the configuration, then I will receive all messages from the logger (without the transformations that your plugin does, since it is disabled).
You need to fix the data transfer to the next pipeline.
Hope you can help me solve this problem!