flyteorg / flyte

Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
https://flyte.org
Apache License 2.0
5.17k stars 550 forks source link

Add support for otlp trace exporters #5477

Closed Sovietaced closed 4 days ago

Sovietaced commented 2 weeks ago

Tracking issue

Closes https://github.com/flyteorg/flyte/issues/5476

Why are the changes needed?

Distributed traces can only be exported using Jaeger with the current code. The jaeger protocol has been deprecated in favor of the open telemetry protocol (OTLP). Additionally, if we're using OTEL client libraries Flyte should also just support their native protocol as well.

We should also deprecate use of the Jaeger exporter and slate it for removal.

What changes were proposed in this pull request?

This pull request adds two new trace exporter strategies: otlpGrpc (OTLP over gRPC) and otlpHttp (OTLP over HTTP).

How was this patch tested?

This patch was tested locally in our non production environment.

Check all the applicable boxes

Related PRs

Docs link

codecov[bot] commented 2 weeks ago

Codecov Report

Attention: Patch coverage is 58.33333% with 5 lines in your changes missing coverage. Please review.

Project coverage is 60.99%. Comparing base (c10346d) to head (f87a1aa).

:exclamation: Current head f87a1aa differs from pull request most recent head 2337fe1

Please upload reports for the commit 2337fe1 to get more accurate results.

Files Patch % Lines
flytestdlib/otelutils/factory.go 54.54% 3 Missing and 2 partials :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #5477 +/- ## ========================================== - Coverage 61.01% 60.99% -0.03% ========================================== Files 794 793 -1 Lines 51441 51335 -106 ========================================== - Hits 31388 31310 -78 + Misses 17161 17139 -22 + Partials 2892 2886 -6 ``` | [Flag](https://app.codecov.io/gh/flyteorg/flyte/pull/5477/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=flyteorg) | Coverage Δ | | |---|---|---| | [unittests-datacatalog](https://app.codecov.io/gh/flyteorg/flyte/pull/5477/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=flyteorg) | `69.31% <ø> (ø)` | | | [unittests-flyteadmin](https://app.codecov.io/gh/flyteorg/flyte/pull/5477/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=flyteorg) | `58.71% <ø> (-0.03%)` | :arrow_down: | | [unittests-flytecopilot](https://app.codecov.io/gh/flyteorg/flyte/pull/5477/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=flyteorg) | `17.79% <ø> (ø)` | | | [unittests-flytectl](https://app.codecov.io/gh/flyteorg/flyte/pull/5477/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=flyteorg) | `67.97% <ø> (-0.12%)` | :arrow_down: | | [unittests-flyteidl](https://app.codecov.io/gh/flyteorg/flyte/pull/5477/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=flyteorg) | `79.04% <ø> (ø)` | | | [unittests-flyteplugins](https://app.codecov.io/gh/flyteorg/flyte/pull/5477/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=flyteorg) | `61.81% <ø> (-0.04%)` | :arrow_down: | | [unittests-flytepropeller](https://app.codecov.io/gh/flyteorg/flyte/pull/5477/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=flyteorg) | `57.32% <ø> (+0.02%)` | :arrow_up: | | [unittests-flytestdlib](https://app.codecov.io/gh/flyteorg/flyte/pull/5477/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=flyteorg) | `65.78% <58.33%> (-0.05%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=flyteorg#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

EngHabu commented 1 week ago

@hamersaw @andrewwdye can you plz review this PR?

Sovietaced commented 4 days ago

Closing in favor of: https://github.com/flyteorg/flyte/pull/5504