Open djaglowski opened 4 hours ago
Pinging code owners:
connector/routing: @jpkrohling @mwear
See Adding Labels via Comments if you do not have permissions to add labels yourself.
@mwear, @jpkrohling, I'm excited to move my PoC forward with the plan proposed above, if it sounds good to you.
Also cc'ing @evan-bradley @TylerHelmuth as I think some version of this might eventually be able to solve cases like https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/32080.
Component(s)
connector/routing
Is your feature request related to a problem? Please describe.
Currently the routing connector only has the ability to route based on resource information. In other words, all logs associated with a given resource will be routed together. (The same is true of metrics and spans as well, but this issue directly addresses logs as a first step.)
Describe the solution you'd like
Add a
context
field to routing table options, with possible valuesresource
(default) orlog
. If log context is specified, execute the OTTL condition or statement as a log statement. Based on the result, route the individual log record appropriately.Importantly, there is an implementation challenge here because appropriate routing at a per-log level requires retention of the higher level scope and resource information, without pulling in other logs as well.
I believe I finally have a workable solution, demonstrated in https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/35939. This solution is based on the addition of helper functions that can "split" a
plog.Logs
according to criteria at either the resource or log level. When splitting at the log level, scope and resource information are retained.I'd like to implement this functionality for logs first, then later for other signals once the strategy proves out. The initial sequence of PRs will look like this:
pkg/pdatautil
or eventually intopdata
, but for now keep them unexported.)Describe alternatives you've considered
No response
Additional context
No response