search

cloudflare / pingora

A library for building fast, reliable and evolvable network services.
Apache License 2.0
22.04k stars 1.22k forks source link

OpenTelemetry with Tracing Integration for HttpProxy #237

Open OshoOS opened 6 months ago

OshoOS commented 6 months ago

What is the problem your feature solves, or the need it fulfills?

It increases the observability through telemetry

Describe the solution you'd like

Integrating with Rust OpenTelemetry, and trace OpenTelemetry.

https://docs.rs/tracing-opentelemetry/latest/tracing_opentelemetry/

Describe alternatives you've considered

What other solutions, features, or workarounds have you considered that might also solve the issue? What are the tradeoffs for these alternatives compared to what you're proposing?

Additional context

github2023spring commented 5 months ago

Currently, is there a way to measure the latency of an upstream request, and instrument it through metrics?

drcaramelsyrup commented 5 months ago

It is possible to measure upstream latency by tracking timestamps and measuring across CTX fields depending on what you're interested in, e.g. saving Instant on connected_to_upstream and comparing to a timestamp in the upstream_response_filter for TTFB since connection establishment.

I think it would be nice to have more user guidance on how one might integrate OpenTelemetry/Jaeger tracing: https://github.com/cloudflare/pingora/issues/258. But it is possible to do today also by saving tracing context in the CTX.