cloudflare / foundations

Cloudflare's Rust service foundations library.
https://blog.cloudflare.com/introducing-foundations-our-open-source-rust-service-foundation-library
BSD 3-Clause "New" or "Revised" License
1.25k stars 51 forks source link

Add option to collect log lines emitted in traced contexts in span logs #58

Open LloydW93 opened 1 month ago

LloydW93 commented 1 month ago

Example use case:

An HTTP service, for which some requests are sampled through distributed tracing, whilst its syslogs are sampled/rate limited in collection - so for a traced request with a failure, it's possible the log information from that error has been dropped, or if it is, exists in some separate system.

Instead, as foundations already provides both a logging and tracing module, the logging module could be configurable to additionally send log entries over to the tracing module. Unlike other drains and pipelines, this would not be heavily filtered and sampled - so the logs that are most useful for the requests we have traced information for are the ones we know we will have retained, right where we need to see them.

inikulin commented 1 month ago

Not sure I understand the motivation here.

syslogs are sampled/rate limited in collection

They are not, unless it's explicitly specified. Most of our services don't have rate limiting for logs. If you want to not rate limit logs if there's active tracing span in scope then more likely most of the hot paths in the service have traces, so, effectively, you just disable rate limiting. So, I'm not sure there's any sense to have this functionality.