[Task]: Add support for custom data for structured logging - Githubissues

apache / beam

Apache Beam is a unified programming model for Batch and Streaming data processing.

https://beam.apache.org/

Apache License 2.0

7.87k stars 4.26k forks source link

[Task]: Add support for custom data for structured logging #26460

Open udim opened 1 year ago

udim commented 1 year ago

What needs to happen?

This issue tracks the implementation of https://s.apache.org/beam-structured-logging. The feature will allow adding custom data fields to log entries.

High-level steps:

Adding a new field in LogEntry (beam_fn_api.proto)
Adding support for structured logging in Java, Python, and Go SDKs.
Adding support for new field in Dataflow, possibly other runners (contributions welcome).

Issue Priority

Priority: 2 (default / most normal work should be filed as P2)

Issue Components

[X] Component: Python SDK
[X] Component: Java SDK
[X] Component: Go SDK
[ ] Component: Typescript SDK
[ ] Component: IO connector
[ ] Component: Beam examples
[ ] Component: Beam playground
[ ] Component: Beam katas
[ ] Component: Website
[ ] Component: Spark Runner
[ ] Component: Flink Runner
[ ] Component: Samza Runner
[ ] Component: Twister2 Runner
[ ] Component: Hazelcast Jet Runner
[ ] Component: Google Cloud Dataflow Runner

udim commented 1 year ago

.take-issue