search

elastic / apm

Elastic Application Performance Monitoring - resources and general issue tracking for Elastic APM.
https://www.elastic.co/apm
Apache License 2.0
384 stars 114 forks source link

Agents re-configure applications to replace plain-text formatters with ECS formatters #373

Open felixbarny opened 3 years ago

felixbarny commented 3 years ago

Users can set a configuration option to make APM Agents automatically re-configure logging for them. After opting into it, agents instrument the construction of log appenders (such as file or stdout appenders) and replace the default plain-text formatter (such as PatternLayout) with the equivalent from the ECS logger that formats the logs to ECS-compliant JSON.

This is an interesting option when manually changing the logging setup of an application is not a trivial task such as when there are long dev release cycles, onboarding logs for legacy applications, or standard off-the-shelf applications. Users still have to manually configure Fleet/Filebeat to ship the logs.

This is only feasible for agents that support some kind of instrumentation. This is the result of an early feasibility check per agent:

Agent Comment
.NET core ❓ 1-liner code change is needed. Potentially doable with no code changes via runtime IL manipulation with profiler
.NET Framework ❌ 1-liner code change needed, Possible for biggest frameworks Serilog and NLog
Go ❌ 3-liner code change needed
Java ✅ POC exists
Node.js ✅ seems feasible
PHP ✅ seems feasible
Python ✅ seems feasible
Ruby ✅ seems feasible

Spec issue

Agent issues

Extend ECS logging documentation

Support replacement of plain-text formatters with ECS formatters

trentm commented 3 years ago

Answering tentatively for Node.js, it seems feasible that we should be able to do this. Each of the logging frameworks that we might support (winston, pino, morgan, bunyan) have their own quirks that might make complexity an issue.

eyalkoren commented 3 years ago

See related spec for more details.