search

open-telemetry / opentelemetry-rust

The Rust OpenTelemetry implementation
https://opentelemetry.io
Apache License 2.0
1.73k stars 393 forks source link

No way to configure `max_{en,de}coding_message_size` of `TraceServiceClient` #1285

Open zohnannor opened 9 months ago

zohnannor commented 9 months ago

Related to #837

Problem

In #1016, tonic was bumped to 0.9 and the methods were added to TraceServiceClient for configuring the max_{en,de}coding_message_size values used by tonic. However, (I assume this was overlooked) there's no way of calling those methods from the opentelemetry_otlp::OtlpExporterPipeline public API, so users are stuck with the default values.

This causes OpenTelemetry trace error occurred. Exporter otlp encountered the following error(s): the grpc server returns error (Some resource has been exhausted): , detailed error message: grpc: received message larger than max (5696602 vs. 4194304) error to happen with no way of fixing it.

Solution

TraceServiceClient created here in 0.20 (the currently most recent published version) with the default values.

https://github.com/open-telemetry/opentelemetry-rust/blob/dfeac078ff7853e7dc814778524b93470dfa5c9c/opentelemetry-otlp/src/span.rs#L391

Now, it is created here

https://github.com/open-telemetry/opentelemetry-rust/blob/08c3d48dc8661773d96b3ab31994db624ac4a8eb/opentelemetry-otlp/src/exporter/tonic/trace.rs#L34

without any (except for send_compressed) configuration, as well.

We should expose a way for users of the library to configure the underlying TraceServiceClient.

While digging in the code, I also noticed that the custom tonic::transport::Channel one can configure for opentelemetry_otlp::OtlpExporterPipeline, is unused in the build_channel implementation.

zohnannor commented 9 months ago

I realized now that the error was coming from the OpenTelemetry collector I was using. So the error mentioned could not be resolved from the application side even if the methods were accessible and max_{en,de}coding_message_size values could be configured. But the issue is still relevant.