drewmiranda-gl
commented
1 year ago Using a lookup table for the stream is an interesting challenge that i'm not sure how to address because the message doesn't appear to contain a stream field until it exists pipeline processing and doesn't appear to be exposed inside of pipelines to be used for lookups :(
My only thought for this is to have the pipeline a message is routed into add the stream name as a normal pipeline rule.
What?
A common requirement is to annotate certain messages with names of various entities, e.g. streams, inputs, forwarders, sidecars, and others. Currently, there's no easy and unified way to get to that information, without going through the API using a token, but that is very expensive for something that is already available to the server that is running the processing.
This feature request is for creating a data adapter that exposes catalog-style metadata using multiple paths:
All entities available to the content pack catalog should be in this lookup table, as well as ephemeral things like running forwarders and sidecars, so users can annotate incoming messages.
Why?
The message detail view shows you the forwarder and input names a message came in one, but there's no way to group on that field and decorate it with the human-readable name. That could be done with a simple lookup decorator (although those are not supported in aggregations yet), or in pipelines with a custom field if the name should be searchable as well, but that information isn't currently available.
Your Environment