[RFC] Add logging to Toolpad

[apedroferreira]

Duplicates

• [X] I have searched the existing issues

Latest version

• [X] I have tested the latest version

Summary 💡

From https://github.com/mui/mui-toolpad/issues/592 Add logging to Toolpad.

What should we log?

• Frontend application logs
• Request tracing
• REST datasource requests

What logging aggregation service should we use, if any? Some popular ones are: Datadog, New Relic, Grafana Loki (more complicated to set up)

Feel free to suggest others.

Examples 🌈

No response

Motivation 🔦

No response

[bytasv]

I'm wondering what exactly we would like to see at this moment? Maybe error logging would be enough for now until we know better?

[apedroferreira]

I'm wondering what exactly we would like to see at this moment? Maybe error logging would be enough for now until we know better?

Maybe the logs Nextjs gives us out of the box are enough to start with. I think that would also include any errors if the application crashes for example.

[Janpot]

I want to be able to get insight into the requests that are made from the backend to external hosts. To be able to gain insight into load and spot abuse.

[bharatkashyap]

What logging service should we use? Some popular ones are: Datadog, New Relic, Grafana Loki (more complicated to set up)

Feel free to suggest others.

https://github.com/winstonjs/winston

[Janpot]

Ideally

1. we log to stdout, the host environment is responsible for collecting logs. This is important for self-hosters.
2. each log line is a json message, in a standard format containing timestamp, message, severity and metadata. I'm familiar with bunyan and pino.
3. logs are searchable, filterable, can be aggregated and dashboards can be built based on metadata. I'm familiar with elasticsearch/kibana for this purpose.

regarding 1. it seems that render.com has more or less an all-or-nothing approach with regards to log shipping: https://render.com/docs/log-streams. This may be a quick and easy solution for us. I'm not quite sure about predicting the cost. alternatively we'll have to ship logs from within the application but I don't want to support that option towards self-hosters. The problems with shipping logs from within the application are:

• they add overhead (pino concentrates on minimizing that)
• they are affected by application. e.g. OOM issues, application crashes, blocked event loops,... can all result in log messages not getting transported and thus result in blind spots. We can mitigate that by shipping logs from stdout. So instead of using the e.g. winston transports, pipe the output of the process to a log shipper. e.g. yarn start | pino-elasticsearch --node http://localhost:9200. (e.g. we put that in the render.yml start command)

regarding 2. maybe not top priority, but towards self hosters we may have to provide an option to format logs. For me it's important that we standardize on log messages (e.g. standard fields to attach errors, req, res objects with their own formatters,...), and be able to attach metadata. I'd like to enforce the log metadata format rigorously (important if we use elsticsearch) so we may have to wrap whatever logging library we use to enforce that.

regarding 3. I've seen powerful things being done with the ELK stack, it has a learning curve though. Alternatively we can look into any of the options render.com proposes

[apedroferreira]

Thanks for the feedback - will switch to pino as it seems to handle overhead better and I'm also more familiar with it - I agree with the rest and I think my initial implementation already follows most of these things, should be just a matter of fine tuning it.

[Janpot]

To clarify, I like pino, but I'm fine with winston if the team prefers. I think the main thing I would change is to remove the log formatting from the app and instead have it output raw jsonline. Then for dev move the formatter to its own process by piping the start command into it. (And for prod we can do the same for now, and replace with an elasticsearch logshipper if we want to add that in the future)