[Feature Request]: OpenTelemetry Support

[Kludex]

What would you like to happen?

This is more a question than a feature request.

I was wondering... Why I don't see any OpenTelemetry related issue on Apache Beam? Is it because the runners already provide observability about their jobs?

I'm more interested in the Python side for now, but if there's no reason, would it make sense to create a opentelemetry-instrumentation-apache-beam package?

Issue Priority

Priority: 3 (nice-to-have improvement)

Issue Components

• [X] Component: Python SDK
• [ ] Component: Java SDK
• [ ] Component: Go SDK
• [ ] Component: Typescript SDK
• [ ] Component: IO connector
• [ ] Component: Beam YAML
• [ ] Component: Beam examples
• [ ] Component: Beam playground
• [ ] Component: Beam katas
• [ ] Component: Website
• [ ] Component: Infrastructure
• [ ] Component: Spark Runner
• [ ] Component: Flink Runner
• [ ] Component: Samza Runner
• [ ] Component: Twister2 Runner
• [ ] Component: Hazelcast Jet Runner
• [ ] Component: Google Cloud Dataflow Runner

[liferoad]

This is a good idea. We do have a plan to investigate this idea. cc @Abacn Meanwhile, you are welcome to work on this.