Error: when start the Exporter

[OjerDev]

Here below is my version details:

Redis: C:\Users>redis-cli -v redis-cli 3.0.504

Python : C:\Users>python Python 3.10.0 (tags/v3.10.0:b494f59, Oct 4 2021, 19:00:18) [MSC v.1929 64 bit (AMD64)] on win32

How I am starting in my windows environment:

Step 1: C:\Users>pip install rq-exporter Requirement already satisfied: rq-exporter in c:\program files\python310\lib\site-packages (1.9.0) Requirement already satisfied: redis>=3.5.3 in c:\program files\python310\lib\site-packages (from rq-exporter) (4.0.1) Requirement already satisfied: rq>=1.8.0 in c:\program files\python310\lib\site-packages (from rq-exporter) (1.10.0) Requirement already satisfied: prometheus-client>=0.8.0 in c:\program files\python310\lib\site-packages (from rq-exporter) (0.12.0) Requirement already satisfied: deprecated in c:\program files\python310\lib\site-packages (from redis>=3.5.3->rq-exporter) (1.2.13) Requirement already satisfied: click>=5.0.0 in c:\program files\python310\lib\site-packages (from rq>=1.8.0->rq-exporter) (8.0.3) Requirement already satisfied: colorama in c:\program files\python310\lib\site-packages (from click>=5.0.0->rq>=1.8.0->rq-exporter) (0.4.4) Requirement already satisfied: wrapt<2,>=1.10 in c:\program files\python310\lib\site-packages (from deprecated->redis>=3.5.3->rq-exporter) (1.13.3)

Step 2: Starting the exporter: C:\Users>rq-exporter C:\Program Files\Python310\lib\site-packages\redis\connection.py:72: UserWarning: redis-py works best with hiredis. Please consider installing warnings.warn(msg) [2021-11-19 15:01:26] [rq_exporter] [ERROR]: Incorrect RQ class location Traceback (most recent call last): File "C:\Program Files\Python310\lib\site-packages\rq_exporter__main__.py", line 241, in main REGISTRY.register(RQCollector(connection, worker_class, queue_class)) File "C:\Program Files\Python310\lib\site-packages\prometheus_client\registry.py", line 26, in register names = self._get_names(collector) File "C:\Program Files\Python310\lib\site-packages\prometheus_client\registry.py", line 66, in _get_names for metric in desc_func(): File "C:\Program Files\Python310\lib\site-packages\rq_exporter\collector.py", line 55, in collect for worker in get_workers_stats(self.worker_class): File "C:\Program Files\Python310\lib\site-packages\rq_exporter\utils.py", line 70, in get_workers_stats workers = worker_class.all() File "C:\Program Files\Python310\lib\site-packages\rq\worker.py", line 123, in all worker_keys = get_keys(queue=queue, connection=connection) File "C:\Program Files\Python310\lib\site-packages\rq\worker_registration.py", line 47, in get_keys return {as_text(key) for key in redis.smembers(redis_key)} File "C:\Program Files\Python310\lib\site-packages\redis\commands\core.py", line 1769, in smembers return self.execute_command('SMEMBERS', name) File "C:\Program Files\Python310\lib\site-packages\redis\client.py", line 1067, in execute_command conn = self.connection or pool.get_connection(command_name, **options) File "C:\Program Files\Python310\lib\site-packages\redis\connection.py", line 1177, in get_connection connection.connect() File "C:\Program Files\Python310\lib\site-packages\redis\sentinel.py", line 45, in connect self.connect_to(self.connection_pool.get_master_address()) File "C:\Program Files\Python310\lib\site-packages\redis\sentinel.py", line 113, in get_master_address master_address = self.sentinel_manager.discover_master( File "C:\Program Files\Python310\lib\site-packages\redis\sentinel.py", line 234, in discover_master masters = sentinel.sentinel_masters() AttributeError: 'Redis' object has no attribute 'sentinel_masters'

This is the error I am facing. Guide me where I did wrong or any pre-requisite thing I need to incomplete?

[mdawar]

Hi,

Sorry about this, it seems that we have introduced a bug in the last version v1.9.0, please use the latest version v1.9.1 that was just published. Please feel free to re-open this issue if you still have any problems after trying the latest version.

Thank you.

[OjerDev]

@mdawar It's working fine on my windows machine. Thank you for the response.

But, I am unable to install the RQ-exporter python package in my AWS-EC2 Linux box.

ubuntu@ip-xxx-xx-xx-xxx:/$ pip install rq-exporter==1.9.1 Collecting rq-exporter==1.9.1 Could not find a version that satisfies the requirement rq-exporter==1.9.1 (from versions: ) No matching distribution found for rq-exporter==1.9.1

ubuntu@ip-xxx-xx-xx-xxx:/$ python --version Python 2.7.17

ubuntu@ip-xxx-xx-xx-xxx:/$ python3 --version Python 3.6.9

ubuntu@ip-xxx-xx-xx-xxx:/$

[mdawar]

@mdawar It's working fine on my windows machine. Thank you for the response.

But, I am unable to install the RQ-exporter python package in my AWS-EC2 Linux box.

ubuntu@ip-xxx-xx-xx-xxx:/$ pip install rq-exporter==1.9.1 Collecting rq-exporter==1.9.1 Could not find a version that satisfies the requirement rq-exporter==1.9.1 (from versions: ) No matching distribution found for rq-exporter==1.9.1

ubuntu@ip-xxx-xx-xx-xxx:/$ python --version Python 2.7.17

ubuntu@ip-xxx-xx-xx-xxx:/$ python3 --version Python 3.6.9

ubuntu@ip-xxx-xx-xx-xxx:/$

This problem could be related to caching.

Can you please try installing using a direct link to the wheel file:

pip install https://files.pythonhosted.org/packages/c1/73/d17738ae5fa7f1197736eee30537411ce5c64a80b118545b4ecdfa231c50/rq_exporter-1.9.1-py3-none-any.whl

If you're wondering, I got the link from the Download Files tab on Pypi.

This is temporary of course, when you try later it should be working fine.

[OjerDev]

I have resolved the above issue. In my Linux box, the default python version is 2.7 so that's why RQ-exporter is not installed with pip. so I have changed the python version to 3.7. then it got worked with the previous command "pip install rq-exporter"

Thanks for your patience @mdawar. Really appreciated.

[OjerDev]

@mdawar Now my exporter is working fine.

But on the collected metrics page, worker metrics and jobs metrics are not showing. What do I need to get queue worker and jobs metrics as well?

I have one running queue in my Redis?

Kindly help me. I am new to the Redis things and Prometheus monitoring...

[mdawar]

@OjerDev Did you enqueue any jobs?

Please check out the Development section on the README, there's a full example of the exporter with scripts that enqueue jobs and workers that process them, also you can find the full Prometheus and Grafana dashboard setup, take a look at the docker-compose.yml file and the docker-compose directory.

You can start a full local development environment with docker-compose up.

[OjerDev]

@mdawar Currently, I am not enqueueing any job.. And i have been told not to use docker developmenmt for this whole setup. so from the start, i am running this package manually.

And another thing, i have one custom queue running in my Redis and i need that queue stats only. i have only the queue name and nothing more.

To achieve this one i need your guidance please..

Adding that, i have done the setup of prometheus and grafana. its working correctly.

[mdawar]

@OjerDev As I remember, you need to enqueue jobs to start seeing the metrics, and in this setup you cannot get the historical queue stats, you only get the current queue metrics only and that's due to how rq provides this data.

It doesn't matter if you're using Docker or not, I was telling you to check the docker-compose setup just to see how things are configured, you can still use all of these configurations without Docker.

Also if you want realtime and accurate stats about the job queue, then this not the right tool for the job, Prometheus scrapes the exporter depending on the configured interval, so anything happening between those scrapes will be missed.

And about the custom queue, you can just use the metrics for this queue only and discard the rest.

[OjerDev]

@mdawar Thanks for clarifying my doubt. Getting the current queue metrics is fine for me. I have checked the custom scripts file in the path and I can see how you are configured and used the custom scripts in the docker-compose.yml file.

Are there any usage instructions for how to use these custom scripts in our exporter?

[mdawar]

@mdawar Thanks for clarifying my doubt. Getting the current queue metrics is fine for me. I have checked the custom scripts file in the path and I can see how you are configured and used the custom scripts in the docker-compose.yml file.

Are there any usage instructions for how to use these custom scripts in our exporter?

Actually you don't use any of these scripts.

Let me clarify a couple of things, you don't need to add any extra code to be able to use this exporter, this exporter must be configured to connect to the Redis instance you're using for rq, that's it.

So you just use rq as you would without this exporter, this exporter runs as a separate service to provide the job/queue metrics for Prometheus to collect them, that's the only purpose of this exporter, and then it's up to you how you use these metrics and display them or create alerts or whatever.

I have provided an example Grafana dashboard to display these metrics but it's in no means a complete solution, it's just provided as an example.