Structured log package in node core

[Raynos]

What is the problem this feature will solve?

To have a structured log package available in stdlib so that all npm modules can log in a single fashion instead of

• logging to console
• not logging
• Having some arbitrary logger interface
• Emitting random log events
• Worst case: they literally depend on Winston and it’s 200 transitive dependencies

What is the feature you are proposing to solve the problem?

Implement something like https://go.dev/blog/slog

What alternatives have you considered?

No response

[NextThread]

Hi, Can I work on this

[MoLow]

You can work on this, but I must give a disclaimer: this kind of feature will probably involve a lot of "soft" work in terms of achieving consensus both on its necessity and then on the implementation. if you are looking for a smooth entry to this project I would choose an issue with less complexity that also has a consensus about how it should be handled

[NextThread]

You can work on this, but I must give a disclaimer: this kind of feature will probably involve a lot of "soft" work in terms of achieving consensus both on its necessity and then on the implementation. if you are looking for a smooth entry to this project I would choose an issue with less complexity that also has a consensus about how it should be handled

yeah sure, please give me an issue with less complexity

[github-actions[bot]]

There has been no activity on this feature request for 5 months and it is unlikely to be implemented. It will be closed 6 months after the last non-automated comment.

For more information on how the project manages feature requests, please consult the feature request management document.

[github-actions[bot]]

There has been no activity on this feature request and it is being closed. If you feel closing this issue is not the right thing to do, please leave a comment.

For more information on how the project manages feature requests, please consult the feature request management document.

[debadree25]

Reopening ref to https://x.com/vinii_joga10/status/1773674350102659178?s=20

[H4ad]

As soon I get some time, I will research how the other langs are handling logs, and I will try to bring some API suggestions on how we can introduce logging at Node.

I think if we create a good abstraction for logging, we can then bring the pino implementation to have some default features, like structured logging, redact, and other features.

/cc @mcollina to bring more ideas on this topic since you know a lot about logging & pino.

[Delapouite]

I will research how the other langs are handling logs

Here's a 2023 article of the official Go blog about this : https://go.dev/blog/slog

[mcollina]

I think we could bring 80% of the pino infrastructure down to core. pino is composed of:

1. public api
2. off thread logging processing (especially useful for network destinstions and heavy processing).
3. main thread log redaction
4. opinionated log format
5. re-do of createWritableStream to maximize speed and handling of overflow situations (disk is busy, what do you want to do?)

It'd need to loose some of the opinions and add some overhead to be more generic, but that can be balanced.

[H4ad]

Hey, it took me a long time to start working on research again as I didn't have a good design API in mind around this logging feature. Initially, I tried to create something from scratch, but then I realized that it would be better to rely on a more mature and stable API, like Pino.

So I created a repository to write a wrapper around Pino that exposes three methods:

• setOptions: To customize the options of the main pin instance.
• getLogger: To get a child instance of the main instance (configured via setOptions).
• buildTransport: Basically exposes a way to create new transports for pino.

I wrote more details about which properties are exposed and how each method behaves in the repository, so I recommend you take a look at the wrapper implementation, which is quite simple (less than 200 lines).

For me, the main advantage of using pino is the solid foundation around the transports that Matteo talks more about in this video.

In terms of extensibility, it is very easy to introduce or let users create new transports using buildTransport, currently the main transport is stdout but we can expose transports to store the logs in file, or send via http and many others.

Please have a look at the repository (link) and tell me what you guys think about this approach, there are a lot of things I don't know yet, like:

• How do we want to include pino in Node, exactly like undici or do we want to completely rewrite it?
• What transports do we want to showcase?
• Are these three methods I exposed enough to create a PR to start this experimental feature? Later we can present the other parts/transports and expose them via node:logging/transports/file.
• Should we expose the redact or should we just remove that part to avoid security issues?

[H4ad]

Also, to advance this proposal, I created a bunch of issues on the nodejs-logging-proposal repository to try to advance the implementation and have more feedback about each aspect, so feel free to open more issues or add comments to each issue.

[mcollina]

I think we should focus on bringing pino in step-by-step. I would also recommend adopting the log levels of https://datatracker.ietf.org/doc/html/rfc5424.

I would remove some of the extreme string mangling.

Imho the steps to bring this to core would:

• [ ] bring in sonic-boom as createWriteStream() is almost unsuitable for logging purposes due to its lack of synchronous output.
• [ ] bring in thread-stream, possibly with a more generic implementation.
• [ ] implement a logging interface itself. Pino is opinionated and maybe we need a slightly less opinionated library, e.g. something that would work similarly to console.log varargs.

[Raynos]

It's possible to have something using varargs, like the go implementation with alternating arguments for key & value. But we would want to have some form of structured output in a text or json format with key/value pairs.

It's the printf style logging with util.format that causes unstructured logs that are not parsable or searchable to be generated.

[H4ad]

@mcollina This means we are using the foundation of pino but not necessarily bringing pino itself, right? Instead of doing like undici, we will have a fork that is less opinionated?

[kibertoad]

maybe core of pino could be extracted, and then used by pino itself?..

[mcollina]

@kibertoad I think more of redo without the cruft accumulated in 8 years + fixing some old warts from the Crankshaftscript era.

[github-actions[bot]]

There has been no activity on this feature request for 5 months. To help maintain relevant open issues, please add the https://github.com/nodejs/node/labels/never-stale label or close this issue if it should be closed. If not, the issue will be automatically closed 6 months after the last non-automated comment. For more information on how the project manages feature requests, please consult the feature request management document.