Open hawkw opened 5 years ago
I'm working on a draft pr in #1345 that addresses this.
Adding a "user story" comment, since I think I am looking for this feature.
I log to stdout, file, and OpenTelemetry.
Because I want the OpenTelemetry frontend (jaeger) to display things correctly, I add an extra field to my span: otel.name = %var
.
But this looks ugly in stdout and file logs, so I would like to not display fields starting with otel.*
there (and not implement a whole new formatter or similar).
So I guess +1 to this feature.
Similar use case to @torsteingrindvik (otel.*
). It would be nice to have more control over the span fields which are logged in the Full
formatter. That's another +1 for this feature.
Feature Request
Crates
tokio-trace-subscriber
tokio-trace-fmt
Motivation
Spans and events may be created with up to 32 fields. In many cases, these fields will provide fairly verbose diagnostic information about the program. Sometimes, users may wish to record data about a span or event, but exclude particularly verbose data attached to it (such as fields provided by libraries).
For example,
tokio-trace-futures
currently adds fields describing the task to spans generated by thespawn!
macro. In the future, thetokio
runtime will likely enrich these spans with additional information about the task's state in the executor. It's likely that this additional information will only be necessary when diagnosing issues involving the executor or the application's interaction with it, so users probably will not typically want to display them by default. Fortunately, the introduction of.
into the permissible field name characters allows "namespacing" these fields; for example, task-specific data is currently in fields with names such astokio.task.$FIELD
.Proposal
I propose adding a utility for wrapping a
Visit
implementation with aFieldFilter
(or similar) struct. This wrapper would implementVisit
by checking if field names match a filter, and if they do not, it would do nothing. Otherwise, it would forward to the wrapped visitor. This could live intokio-trace-subscriber
, as it's primarily a tool for subscriber implementors to use.Additionally, we'd have functions for constructing a "field filter" from various types, such as a
Fn(&Field) -> bool
, and a list of allowed or denied patterns.As a proposal, I suggest a grammar for constructing allow/deny lists from strings similar to the
EnvFilter
intokio-trace-fmt
. A single filter pattern would consist of:tokio.task.is_spawned
,message
,user.email
*
should match up to the next.
character.tokio.task.*
would match any field name beginning withtokio.task.
followed by one more Rust identifiertokio.*.is_spawned
: matches any field name beginning withtokio.
, followed by a single rust identifier segment, followed by.is_spawned
tokio.task.is_*
: matchestokio.task.is_spawned
,tokio.task.is_blocking
,tokio.task.is_parked
, and so on.*.*.*
matches any field name consisting of exactly three dotted segments**
) would match any number of characters.tokio.**
would matchtokio.task.is_spawned
,tokio.runtime
,tokio.task.state.is_blocked
, and so on**.is_spawned
: matchestokio.task.is_spawned
,romio.is_spawned
,foo.executor.state.is_spawned
,std.thread.is_spawned
, and so on**
: matches anythingWe could parse an allow/deny list from a comma separated list of the above patterns.
Additionally, we could add syntax to filters that are dependent on the name of the span/event the field is on, as well as the field name itself. I think that should probably be a follow-up issue, since it requires an API for informing the visitor what the span's name is.
Alternatives
Rather than implementing our own globbing syntax, we could use regular expressions. This might be simpler to implement, since we can use existing regex libraries, and would allow more flexible filters.
However, since we have a much more restricted grammar for field names than "all UTF-8 characters", there are advantages for not allowing full regular expressions. For example, since
,
is not a valid field name character, we can parse a list of patterns from a comma-separated string. However, if we accept arbitrary regexen,,
may be part of a regex, rather than a separator between regexen.Similarly,
.
is a special character with a defined meaning in most regex implementations. This means that regexen that matchtokio-trace
field names with.
s would need to escape every.
character. This would lead to regexen likefoo\..+.bar
, which is awkward and hard to parse.Finally, it would be possible to construct a regex that matches only field names that are invalid and will never occur. Allowing such a regex and testing all field names against it seems inefficient. Using our own globbing syntax would allow us to reject patterns that will never match a field.