How do I monitor a stream?

[BurningDog]

Describe the problem

Firstly, thank you for your work on this repo!

Using Grafana Cloud, I am trying to monitor a stream in redis called messages, but can't access any stream-related metrics in Grafana Cloud.

The other redis metrics are coming through to Grafana Cloud just fine. I can run the following commands on the redis cli:

> XLEN messages
(integer) 0

> XINFO STREAM messages
1) "length"
2) "0"
3) "radix-tree-keys"
4) "0"
5) "radix-tree-nodes"
6) "1"
7) "last-generated-id"
8) "1699976764920-0"
9) "max-deleted-entry-id"
10) "1699976764920-0"
11) "entries-added"
12) "279"
13) "recorded-first-entry-id"
14) "0-0"
15) "groups"
16) "1"
17) "first-entry"
18) "null"
19) "last-entry"
20) "null"

> XINFO CONSUMERS messages symfony
1) 1) "name"
   2) "consumer-1"
   3) "pending"
   4) "0"
   5) "idle"
   6) "91440756"
   7) "inactive"
   8) "91532627"
2) 1) "name"
   2) "consumer-2"
   3) "pending"
   4) "0"
   5) "idle"
   6) "895"
   7) "inactive"
   8) "16619247"

The config I am using in the Grafana cloud agent config file is:

  redis_exporter:
    enabled: true
    redis_addr: "redis://redis:6379"
    redis_user: ${REDIS_MONITOR_USER}
    redis_password: ${REDIS_MONITOR_PASSWORD}
    incl_system_metrics: true
    check_single_streams: messages

The README says that check_single_streams is a comma separated list of streams. I have also tried:

• check_single_streams: db0=messages
• check-streams: db0=messages

What version of redis_exporter are you running?

1.54.0 running inside version 0.37.2 of the Grafana agent.

Running the exporter

The grafana agent is running inside a docker container and is working fine - it's just the stream data that is missing.

Screenshots

In Grafana cloud I can access a bunch of redis metrics:

But nothing with stream:

And nothing with messages:

Additional context

I've read every stream-related issue I could find in this repo, googled the problem, read the docs both here and at https://grafana.com/docs/agent/latest/static/configuration/integrations/redis-exporter-config/ and I can't figure out what I'm doing wrong.

[oliver006]

I have very little to no experience with monitoring streams, that was all contributed by other people so I might not be the right person to help. Have you tried running with --debug to see if there's anything useful there? Maybe the parsing fails silently.

[BurningDog]

Thanks, that did the trick! It gave me enough information to figure out what I was doing wrong.

I can now see both streams and messages in Grafana:

The config which worked was check_single_streams: db0=messages.

The current README describes check-single-streams as:

Comma separated list of streams to export info about streams, groups and consumers. The streams specified with this flag will be looked up directly without any glob pattern matching. Use this option if you don't need glob pattern matching; it is faster than check-streams.

Whereas for check-streams the README says:

Comma separated list of stream-patterns to export info about streams, groups and consumers. Syntax is the same as check-keys.

I would strongly suggest changing the README for check-single-streams to include both the mention of check-keys and an example, like so:

check-single-streams:

Comma separated list of streams to export info about streams, groups and consumers. The streams specified with this flag will be looked up directly without any glob pattern matching. Use this option if you don't need glob pattern matching; it is faster than check-streams. Syntax is the same as check-keys e.g. to check a stream called messages use db0=messages.

[BurningDog]

And just to show some metrics...this shows the number of items in the messages stream:

I've set the collector to run once every 60 seconds - and it turns out the Redis metrics don't roll over i.e. I dispatched 3 other sets of jobs after the spike in the graph, each of them creating 13 items in the streams...and they didn't show up at all.

[oliver006]

Very good! And thanks for explaining the details If you think the docs can use more detail around this then please go ahead and open a PR. Otherwise people can also find this by searching old issues.