open-telemetry / opentelemetry-rust

The Rust OpenTelemetry implementation
https://opentelemetry.io
Apache License 2.0
1.82k stars 422 forks source link

Avoid string copy in tracing appender #1921

Open cijothomas opened 2 months ago

cijothomas commented 2 months ago

Tracing appender's EventVisitor currently copies the &str to an owned string in the heap. This is the case even when the user provides 'static str to the tracing macros. Opening an issue to see if this can be improved. Most likely this requires some support from tracing itself. Alternatively, this could be optimized for the sync exporters (like etw/user_events) alone, by doing serialization in the exporter in "streaming fashion", without any intermediate storage.

cijothomas commented 1 month ago

https://github.com/microsoft/rust_win_etw/blob/main/win_etw_tracing/src/lib.rs#L343 Same alloc.

lalitb commented 3 weeks ago

Current update on the options tried :

This get's complicated to support. The generic solution would be adding &str variant type to StringValue, however adding any such lifetime specifier propagates through function signatures, struct definitions, and trait implementations, increasing the complexity of the codebase. In this case, the lifetime specifier needs to be added to StringValue -> OtelString ->AnyValue -> LogRecord, and then further to LogRecord::body(), LogRecord::add_attributes(), LogRecord::add_attribute(), LogProcessor::emit, and LogExporter::export methods as tried here - https://github.com/open-telemetry/opentelemetry-rust/compare/main...lalitb:opentelemetry-rust:strref-support.

Another option could be additional support of the direct streaming to exporter, without intermediate storage, however this wouldn't support the updates through LogProcessor chaining.

lalitb commented 1 week ago

Have opened a discussion with the tracing folks - https://github.com/tokio-rs/tracing/discussions/3073. Also with the trait definition for the tacing::field::Visit::record_str method, it is not possible to extend the lifetime of str reference without unsafe keyword. Even though using unsafe is a safe operation, as the lifetime of the string is valid for the scope of execution of the tracing macro.

trait tracing::field::Visit  {
fn record_str(&mut self, field: &tracing_core::Field, value: &str);
}