Unable to detect 'sketch' data types in vector vrl language

[seanlowcy77]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

Hi Vector team,

I currently have distribution metrics coming from a DD agent metrics source. I wanted to route these metrics within vector. However I noticed they have tags like "sketch": { "sketch": { "AgentDDSketch": {

When I try to route them via vrl with a condition like '.type != sketch' it doesnt work. This seems to be a bug as mentioned by @jszwedko.

This was also previously discussed in https://discord.com/channels/742820443487993987/1156386370160443462/1156386370160443462

Configuration

Config:

[transforms.route_post_aggr_metrics]
type = "route"
inputs = ["aggregate_metrics"]
route.to_add_vector_hostname = '.type != "sketch"'

Unit test:

[[tests.inputs]]
insert_at = "route_post_aggr_metrics"
type = "metric"

[tests.inputs.metric]
kind = "incremental"
namespace = "XXX"
name = "YYY"
sketch = { sketch = { AgentDDSketch = {bins = {k = [1], n = [1]}, count = 1,min = 1, max = 1, sum = 1, avg = 1}}}

[[tests.outputs]]
extract_from = "route_post_aggr_metrics._unmatched"

[[tests.outputs.conditions]]
type = "vrl"
source = """
assert_eq!(.namespace, "XXX")
assert_eq!(.name, "YYY")

### Version

vector 0.32.2

### Debug Output

```text
checks for transforms ["route_post_aggr_metrics._unmatched"] failed: no events 
received. Topology may be disconnected or transform is missing inputs.
I guess implying that no events are going through ._unmatched (indicating that its going through to_add_vector_hostname)

Example Data

No response

Additional Context

No response

References

https://discord.com/channels/742820443487993987/1156386370160443462/1156386370160443462

[neuronull]

Reproduced this. I would have expected the following to work:

[transforms.route_post_aggr_metrics]
type = "route"
inputs = ["aggregate_metrics"]

[transforms.route_post_aggr_metrics.route.to_add_vector_hostname]
type = "vrl"
source = """
exists(.sketch)
"""

[[tests]]
name = "foo"

[[tests.inputs]]
insert_at = "route_post_aggr_metrics"
type = "metric"

[tests.inputs.metric]
kind = "incremental"
namespace = "XXX"
name = "YYY"
sketch = { sketch = { AgentDDSketch = {bins = {k = [1], n = [1]}, count = 1,min = 1, max = 1, sum = 1, avg = 1}}}

[[tests.outputs]]
extract_from = "route_post_aggr_metrics.to_add_vector_hostname"

[[tests.outputs.conditions]]
type = "vrl"
source = """
assert_eq!(.namespace, "XXX")
assert_eq!(.name, "YYY")
"""

Running tests
test foo ... failed

failures:

test foo:

checks for transforms ["route_post_aggr_metrics.to_add_vector_hostname"] failed: no events received. Topology may be disconnected or transform is missing inputs.

[jszwedko]

@seanlowcy77 it actually looks like type for sketches is agent dd sketch. The following seems to work:

[transforms.route_post_aggr_metrics]
type = "route"
inputs = ["aggregate_metrics"]

[transforms.route_post_aggr_metrics.route.to_add_vector_hostname]
type = "vrl"
source = """
.type == "agent dd sketch"
"""

[[tests]]
name = "foo"

[[tests.inputs]]
insert_at = "route_post_aggr_metrics"
type = "metric"

[tests.inputs.metric]
kind = "incremental"
namespace = "XXX"
name = "YYY"
sketch = { sketch = { AgentDDSketch = {bins = {k = [1], n = [1]}, count = 1,min = 1, max = 1, sum = 1, avg = 1}}}

[[tests.outputs]]
extract_from = "route_post_aggr_metrics.to_add_vector_hostname"

[[tests.outputs.conditions]]
type = "vrl"
source = """
assert_eq!(.namespace, "XXX")
assert_eq!(.name, "YYY")
"""