search

emit-rs / emit

Developer-first diagnostics for Rust applications
Apache License 2.0
123 stars 2 forks source link

Improve performance of OTLP emitter #79

Closed KodrAus closed 1 month ago

KodrAus commented 1 month ago

emit_otlp is currently able to emit an event in ~9700ns based on the simple throughput test I added recently. That's almost 9x slower than using the OpenTelemetry SDK directly (that is, producing owned LogRecords and emitting them through a Logger) so there's some investigation to be done there. It's not an apples-to-apples comparison, but that's also kind of the point. We haven't done a holistic performance investigation of emit yet, so there are probably a lot of low-hanging fruit to explore.

KodrAus commented 1 month ago

Started digging in to this, and a lot of the cost comes from the fact that emit_otlp serializes events into protobuf fragments on-thread, then ships those off to the background worker. As far as I can tell, the OpenTelemetry SDK defers this work until later. Since I'm just blasting events through the pipeline, the OpenTelemetry SDK is discarding most of them, so only serializes and ships a fraction of them.

This scenario is very artificial, so doesn't warrant further investigation I think.