Closed choyuri closed 6 years ago
Hi, that's strange. Could you please show output of rabbitmq-plugins list -v
?
[E*] accept
Version: 0.3.3
Description: Accept header(s) for Erlang/Elixir
[e*] amqp_client
Version: 3.6.11
Description: RabbitMQ AMQP Client
[e*] cowboy
Version: 1.0.4
Dependencies: [cowlib]
Description: Small, fast, modular HTTP server.
[e*] cowlib
Version: 1.0.2
Description: Support library for manipulating Web protocols.
[E*] prometheus
Version: 3.4.0
Description: Prometheus.io client in Erlang
[E*] prometheus_httpd
Version: 2.1.4
Dependencies: [prometheus,accept]
Description: Prometheus.io inets httpd exporter
[ ] prometheus_process_collector
Version: 1.1.0
Description: Prometheus.io process collector
Collector exports the current state of process metrics including cpu, memory,
file descriptor usage and native threads count as well as the process start and up times.
[E*] prometheus_rabbitmq_exporter
Version: v3.6.12.1
Dependencies: [rabbitmq_management,prometheus]
Description: RabbitMQ Prometheus.io metrics exporter
[ ] rabbitmq_amqp1_0
Version: 3.6.11
Dependencies: [amqp_client]
Description: AMQP 1.0 support for RabbitMQ
[E*] rabbitmq_auth_backend_http
Version: 3.6.7
Dependencies: [amqp_client]
Description: RabbitMQ HTTP Authentication Backend
[ ] rabbitmq_auth_backend_ldap
Version: 3.6.11
Description: RabbitMQ LDAP Authentication Backend
[ ] rabbitmq_auth_mechanism_ssl
Version: 3.6.11
Description: RabbitMQ SSL authentication (SASL EXTERNAL)
[ ] rabbitmq_consistent_hash_exchange
Version: 3.6.11
Description: Consistent Hash Exchange Type
[E*] rabbitmq_event_exchange
Version: 3.6.11
Description: Event Exchange Type
[ ] rabbitmq_federation
Version: 3.6.11
Dependencies: [amqp_client]
Description: RabbitMQ Federation
[ ] rabbitmq_federation_management
Version: 3.6.11
Dependencies: [rabbitmq_management,rabbitmq_federation]
Description: RabbitMQ Federation Management
[ ] rabbitmq_jms_topic_exchange
Version: 3.6.11
Description: RabbitMQ JMS topic selector exchange plugin
[E*] rabbitmq_management
Version: 3.6.11
Dependencies: [amqp_client,cowboy,cowlib,rabbitmq_web_dispatch,
rabbitmq_management_agent]
Description: RabbitMQ Management Console
[e*] rabbitmq_management_agent
Version: 3.6.11
Description: RabbitMQ Management Agent
[ ] rabbitmq_management_visualiser
Version: 3.6.11
Dependencies: [rabbitmq_management]
Description: RabbitMQ Visualiser
[ ] rabbitmq_mqtt
Version: 3.6.11
Dependencies: [amqp_client]
Description: RabbitMQ MQTT Adapter
[ ] rabbitmq_recent_history_exchange
Version: 3.6.11
Description: RabbitMQ Recent History Exchange
[ ] rabbitmq_sharding
Version: 3.6.11
Description: RabbitMQ Sharding Plugin
[ ] rabbitmq_shovel
Version: 3.6.11
Dependencies: [amqp_client]
Description: Data Shovel for RabbitMQ
[ ] rabbitmq_shovel_management
Version: 3.6.11
Dependencies: [rabbitmq_management,rabbitmq_shovel]
Description: Management extension for the Shovel plugin
[e*] rabbitmq_stomp
Version: 3.6.11
Dependencies: [amqp_client]
Description: RabbitMQ STOMP plugin
[ ] rabbitmq_top
Version: 3.6.11
Dependencies: [amqp_client,rabbitmq_management]
Description: RabbitMQ Top
[ ] rabbitmq_tracing
Version: 3.6.11
Dependencies: [rabbitmq_management]
Description: RabbitMQ message logging / tracing
[ ] rabbitmq_trust_store
Version: 3.6.11
Description: Client X.509 certificates trust store
[e*] rabbitmq_web_dispatch
Version: 3.6.11
Dependencies: [cowboy]
Description: RabbitMQ Web Dispatcher
[ ] rabbitmq_web_mqtt
Version: 3.6.11
Dependencies: [cowboy,rabbitmq_mqtt]
Description: RabbitMQ MQTT-over-WebSockets adapter
[ ] rabbitmq_web_mqtt_examples
Version: 3.6.11
Dependencies: [rabbitmq_web_dispatch,rabbitmq_web_mqtt]
Description: Rabbit WEB-MQTT - examples
[E*] rabbitmq_web_stomp
Version: 3.6.11
Dependencies: [cowboy,sockjs,rabbitmq_stomp]
Description: Rabbit WEB-STOMP - WebSockets to Stomp adapter
[ ] rabbitmq_web_stomp_examples
Version: 3.6.11
Dependencies: [rabbitmq_web_dispatch,rabbitmq_web_stomp]
Description: Rabbit WEB-STOMP - examples
[e*] sockjs
Version: 0.3.4
Description: SockJS
Thanks!
More questions:
prometheus_registry:collectors(default).
?yes checked on all three. what happened was.. I did a test deploy on node1.. didnt see rabbit metrics tried node2 same issue tried node 3 same issue
thanks
node1: [prometheus_boolean,prometheus_counter,prometheus_gauge,prometheus_histogram, prometheus_mnesia_collector,prometheus_summary, prometheus_vm_memory_collector,prometheus_vm_statistics_collector, prometheus_vm_system_info_collector]
node2: [prometheus_boolean,prometheus_counter,prometheus_gauge,prometheus_histogram, prometheus_mnesia_collector,prometheus_summary, prometheus_vm_memory_collector,prometheus_vm_statistics_collector, prometheus_vm_system_info_collector]
node3:
[prometheus_boolean,prometheus_counter,prometheus_gauge,prometheus_histogram, prometheus_mnesia_collector,prometheus_summary, prometheus_vm_memory_collector,prometheus_vm_statistics_collector, prometheus_vm_system_info_collector]
So rabbitmq collectors just not registered for some reason...
Could you please now try this call prometheus_registry:register_collector(prometheus_rabbitmq_overview_collector).
and look at /metrics
again?
wooh! that worked
ok, so the problem is that rabbitmq_ collectors aren't registered automatically on node startup...
Did you strip debug_info? What if you restart a node?
Anyway will fix registering soon. Nice catch :-)
thank you for debugging!
so i just followed the documentation and performed enable plugin steps only it's a pretty hefty cluster so takes non trivial amount of time for each node to bounce and rejoin :) so was trying to avoid restarting the workaround is great in the meantime.
thanks again!
Should be fixed here: https://github.com/deadtrickster/prometheus_rabbitmq_exporter/releases/tag/rabbitmq-3.6.14.1.
On a 3-node clustered rabbitmq version 3.6.11, I enabled prometheus plugins on all three nodes I see all the beam metrics, but not any of the rabbitmq metrics
Initially tried prometheus_rabbitmq_exporter v3.6.12.1
then tried prometheus_rabbitmq_exporter v3.6.9.1
there are no errors in rabbit logs. prometheus metrics are working.. just missing the rabbitmq metrics
thank you