vectordotdev / vector

A high-performance observability data pipeline.
https://vector.dev
Mozilla Public License 2.0
17.48k stars 1.53k forks source link

prometheus_scrape unable to parse TYPE target info messages #18769

Open rhishikeshj opened 11 months ago

rhishikeshj commented 11 months ago

A note for the community

Problem

If the metrics stream contains messages of the form

# TYPE target info

Vector seems to error out with the following logs

2023-10-04T07:50:41.194651Z ERROR source{component_kind="source" component_id=better_stack_prometheus_source_A1VE5YLVHt1B9KLtumGHREaT component_type=prometheus_scrape component_name=better_stack_prometheus_source_A1VE5YLVHt1B9KLtumGHREaT}: vector::internal_events::prometheus: Parsing error. url=http://localhost:9404/metrics error=WithLine { line: "# TYPE target info", kind: InvalidMetricKind { input: "info" } } error_type="parser_failed" stage="processing" internal_log_rate_limit=true

And this also seems to prevent any further processing of the scrape.

target info is a standard metric message type according to the OpenSpecification on Info

Configuration

[sources.better_stack_prometheus_source_token]
type = "prometheus_scrape"
endpoints = ["http://localhost:9404/metrics"]
instance_tag = "instance"
scrape_interval_secs = 15

[transforms.better_stack_prometheus_transform_token]
type = "remap"
inputs = [ "better_stack_prometheus_filter_transform_token" ]
drop_on_error = true
source = '''
.dt = del(.timestamp)
'''

[sinks.better_stack_http_sink_token]
type = "http"
inputs = [ "better_stack_prometheus_transform_token" ]
uri = "https://in.logs.betterstack.com/metrics"
method = "post"
encoding.codec = "json"
auth.strategy = "bearer"
auth.token = "token"

Version

vector 0.33.0 (aarch64-unknown-linux-gnu 89605fb 2023-09-27 14:18:24.180809939)

Debug Output

No response

Example Data

No response

Additional Context

No response

References

No response

neuronull commented 11 months ago

Hello 👋

It looks like the Info type is not listed in the Prometheus client library spec as a supported type: https://prometheus.io/docs/concepts/metric_types/

I do see in the code these are the metric types we support (this is also where that error is generated) https://github.com/vectordotdev/vector/blob/ed97f0d862d2b8c9e606d587e03fd1467159c608/lib/prometheus-parser/src/line.rs#L307-L316