How to monitor redis with prometheus ?

[Alexvianet]

For example, we have deployment with:

Instance                                              Process State  AZ   IPs  
cf-redis-broker/84a0e577-4d14-4712-b376-7f2b3e13ba66  running        AZ1  10.1.1.1  
dedicated-node/0a2687b5-4a8d-4321-a36c-a2fd5052c18a   running        AZ2  10.1.1.2
dedicated-node/0dfd7963-c700-4f0a-a2fa-3d218949c3fc   running        AZ2  10.1.1.3
dedicated-node/0eb7ca0e-51c3-412c-a4ea-e1a8c3f8a2be   running        AZ3  10.1.1.4

What is the best practice of redis monitoring:

By the reason of https://github.com/pivotal-cf/cf-redis-release/blob/master/jobs/dedicated-node/spec: redis.security.require_pass: description: Require clients to issue AUTH PASSWORD before processing any other commands. This will be overwritten everytime the broker allocates it as in use.

redis pass is changing each time, end redis_exporter lost connection with:

time="2019-05-09T19:20:49Z" level=debug msg="Trying DialURL(): redis://127.0.0.1:6379"
time="2019-05-09T19:20:49Z" level=debug msg="DialURL() failed, err: ERR invalid password"

Help to resolve the issue, Thanks

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this. Unfortunately, the Pivotal Tracker project is private so you may be unable to view the contents of the story.

The labels on this github issue will be updated when the story is started.

[MirahImage]

Hi @Alexvianet, I think you have a misunderstanding of what it means to allocate an instance. An instance is allocated by cf create-service p-redis <plan-name> <service-name>, then used repeatedly, until de-provisioned by the cf app developer by calling cf delete-service <service-name>.

So long as a service instance is not deleted, the broker will not interact with the service instance, therefore the password will not change. Deprovisioning an instance and creating a new instance, however, will result in a password rotation (this is the expected behaviour).

For monitoring, logs and metrics are emitted through the loggregator system. For more information about how to connect to the loggregator endpoints, please see their documentation: https://docs.cloudfoundry.org/loggregator/log-ops-guide.html

Best, Mirah & @jplebre

[Alexvianet]

Hi @MirahImage loggregator help to monitor application but i need to monitor redis node info: https://redis.io/commands/info

[Alexvianet]

@MirahImage
Please add me to story AlexanderGolubov @alexandergolubov Alexander.Golubov@solera.com

[MirahImage]

Sorry, I had a misunderstanding and have realized this is in relation to the deprecated dedicated VM plan. As is documented in the README, this is deprecated and there is no further feature development on this.

[Alexvianet]

@MirahImage So if i update redis to latest version and remove deprecated dedicated VM plan: requirepass in /var/vcap/store/redis/redis.conf will not changed any more ?

[jplebre]

@Alexvianet the upcoming feature will ship redis-info into syslog and loggregator so the service-instance info is available to app devs.

We will not be changing the requirepass behaviour. As @MirahImage replied above, Deprovisioning an instance and creating a new instance, however, will result in a password rotation (this is the expected behaviour). This is the default behaviour and ensures any new SI provisioned is secure and not reusing creds that have previously been deployed through the foundation.

The service-instance will not be interacted in any way regarding the requirepass for the lifetime of the instance. But provisioning a new instance - even if it has the same service-instance name - will give a brand new set of credentials.