[Metricbeat] RabbitMQ module - not working with version 3.10

[leweafan]

Metricbeat can not collect event.dataset "rabbitmq.node" for RabbitMQ v3.10.

Metricbeat returns error.message:

error in mapping: error applying schema: 3 errors: key `queue_index_journal_write_count` not found; key `io_file_handle_open_attempt_avg_time` not found; key `io_file_handle_open_attempt_count` not found

Using curl request (don't forget to change hostname to real hostname)

curl -u guest -X GET http://localhost:15672/api/nodes/rabbit@hostname | jq '.|keys'

we get all metric keys

[
  "applications",
  "auth_mechanisms",
  "channel_closed",
  "channel_closed_details",
  "channel_created",
  "channel_created_details",
  "cluster_links",
  "config_files",
  "connection_closed",
  "connection_closed_details",
  "connection_created",
  "connection_created_details",
  "context_switches",
  "context_switches_details",
  "contexts",
  "db_dir",
  "disk_free",
  "disk_free_alarm",
  "disk_free_details",
  "disk_free_limit",
  "enabled_plugins",
  "exchange_types",
  "fd_total",
  "fd_used",
  "fd_used_details",
  "gc_bytes_reclaimed",
  "gc_bytes_reclaimed_details",
  "gc_num",
  "gc_num_details",
  "io_read_avg_time",
  "io_read_avg_time_details",
  "io_read_bytes",
  "io_read_bytes_details",
  "io_read_count",
  "io_read_count_details",
  "io_reopen_count",
  "io_reopen_count_details",
  "io_seek_avg_time",
  "io_seek_avg_time_details",
  "io_seek_count",
  "io_seek_count_details",
  "io_sync_avg_time",
  "io_sync_avg_time_details",
  "io_sync_count",
  "io_sync_count_details",
  "io_write_avg_time",
  "io_write_avg_time_details",
  "io_write_bytes",
  "io_write_bytes_details",
  "io_write_count",
  "io_write_count_details",
  "log_files",
  "mem_alarm",
  "mem_calculation_strategy",
  "mem_limit",
  "mem_used",
  "mem_used_details",
  "metrics_gc_queue_length",
  "mnesia_disk_tx_count",
  "mnesia_disk_tx_count_details",
  "mnesia_ram_tx_count",
  "mnesia_ram_tx_count_details",
  "msg_store_read_count",
  "msg_store_read_count_details",
  "msg_store_write_count",
  "msg_store_write_count_details",
  "name",
  "net_ticktime",
  "os_pid",
  "partitions",
  "proc_total",
  "proc_used",
  "proc_used_details",
  "processors",
  "queue_created",
  "queue_created_details",
  "queue_declared",
  "queue_declared_details",
  "queue_deleted",
  "queue_deleted_details",
  "queue_index_read_count",
  "queue_index_read_count_details",
  "queue_index_write_count",
  "queue_index_write_count_details",
  "ra_open_file_metrics",
  "rates_mode",
  "run_queue",
  "running",
  "sockets_total",
  "sockets_used",
  "sockets_used_details",
  "type",
  "uptime"
]

As you can see 3 metrics are missing:

• queue_index_journal_write_count
• io_file_handle_open_attempt_avg_time
• io_file_handle_open_attempt_count

Cause of this issue event.dataset "rabbitmq.node" doesn't return any metrics at all.

According to https://github.com/rabbitmq/rabbitmq-server/pull/4431 metrics were deleted cause been proven irrelevant.

There is a PR for RabbitMQ where some disk metrics were made optional https://github.com/elastic/beats/pull/6887 may be this ones should be too.

[immaro]

+1 for this.

These metrics have been removed from rabbitmq in https://github.com/rabbitmq/rabbitmq-server/pull/4431

[elasticmachine]

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

[botelastic[bot]]

Hi! We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

[aglowienka]

👍🏻

[Joren-Thijs-KasparSolutions]

Any updates on this?

I am currently seeing the same thing with RabitMQ 3.11 while using the latest Metricbeat version v7.17.18

[VaggelisEy8ymioy]

Hello, Do we have any updates on this?

[Linu-Elias]

This issue has been fixed in the latest Metricbeat version v8.15.0, and we no longer get the error mentioned above.