metrics not published on /metrics url

[bizzyman7]

Hi, Just a help wanted issue I believe.

I'm attempting to convert over to prometheus from graphite and starting small.

I'm sending metric:

echo "stats.test.Testing123.10.10.12.26.avg.server1.System.Processes.mysqld.cpu 15.0 1682733244" | nc -q "0" 10.113.54.90 9109

With debug on, I see a message like this from the exporter:

ts=2023-04-29T04:55:06.943Z caller=collector.go:183 level=debug msg="Processing sample" sample="collector.graphiteSample{OriginalName:\"stats.test.Testing123.10.10.12.26.avg.server1.System.Processes.mysqld.cpu\", Name:\"fmc_prcocess_cpu_pct\", Labels:prometheus.Labels{\"job\":\"Testing123\", \"process\":\"mysqld\"}, Help:\"Graphite metric fmc_prcocess_cpu_pct\", Value:12.0, Type:2, Timestamp:time.Date(2023, time.April, 29, 4, 34, 45, 0, time.Local)}"

Config:

- match: stats.*.*.*.*.*.*.avg.server1.System.*.*.cpu
  name: fmc_prcocess_cpu_pct
  labels:
    job: $2
    process: $8

However I never this metric on http://localhost:9108/metrics

Am I doing something wrong?

[matthiasr]

The problem is that your timestamp is too far in the past. Note how the log line is emitted at 04:55 while the timestamp is 04:34.

Prometheus and Graphite differ in their model with time; while Graphite samples are "at that particular time", Prometheus is much more about the now. On the other hand, in Graphite when you don't send a sample for a time window, there's no data at that time, while Prometheus (clients) generally hold the last value.

This exporter reconciles that by holding the last value it has seen for Prometheus to scrape, but dropping the metric if it hasn't gotten a newer sample within 5 minutes. This is configurable with the --graphite.sample-expiry flag. In your case, the exporter accepted a sample that was 11 minutes old and immediately expired it. We could probably log a clearer message in this case… anyway I hope this explanation is helpful.