search

noxdafox / rabbitmq-cloudwatch-exporter

RabbitMQ Plugin for publishing cluster metrics to AWS CloudWatch
Mozilla Public License 2.0
41 stars 9 forks source link

Support more fine-grained metrics publishing filter #16

Closed noxdafox closed 4 years ago

noxdafox commented 4 years ago

Currently, several users are struggling to configure the plugin to contain CloudWatch costs.

A more fine-grained mechanism to configure which metrics to be published is required.

The following is a proposal for a new configuration syntax which is more namespace-based.

# A metric group needs to be enabled for export
# If no other option is specified, everything from the group is exported
cloudwatch_exporter.metrics.overview.enable = true

cloudwatch_exporter.metrics.overview.enable = true
# If types of a specific group are provided, only metrics from those types are exported
cloudwatch_exporter.metrics.overview.export_metrics.1 = "Messages"
cloudwatch_exporter.metrics.overview.export_metrics.2 = "MessagesReady"

cloudwatch_exporter.metrics.queue.enable = true
# For exchange, queue, connection and channel groups, it is possible to control the names of the entities to be published via regular expressions.
cloudwatch_exporter.metrics.queue.export_regex = "^(?!amq.gen-.*).*$"

# It is possible to combine both filters
cloudwatch_exporter.metrics.exchange.enable = true
cloudwatch_exporter.metrics.exchange.export_regex = "foo.*|.*bar"
cloudwatch_exporter.metrics.exchange.export_metrics.1 = "PublishIn"

TODO: provide equivalent in rabbitmq.config old configuration format.

Pilskalns commented 4 years ago

The proposed config looks very powerful, allowing both - very flexible and granular control over pushed metrics. I couldn't wish anything more here.

Pilskalns commented 4 years ago

Hi, I hope you are staying safe these days.

Any estimate when this might be implemented?

noxdafox commented 4 years ago

Hello,

code and documentation are ready. I am planning to include the changes with a major release which will include other features as well.

You can test the new plugin by building the v1 branch. It should be pretty trivial to do so, just follow the README instructions making sure the version of Erlang is the same as your cluster.

Pilskalns commented 4 years ago

Hi,

Unfortunately, my experience with RabbitMQ itself is pretty minimal. I tried to understand the list of dependencies and environment needed to build from sources, there are a bunch of them I am not familiar with. Looks like it can become a rabbit hole (pun intended)...

If you have ready plugin files from this dev version, I would be willing to drop them on our staging environment and give it a test ride and see how it works over a longer period of time.

chathsuom commented 4 years ago

Hi, Any plan to release this sooner?

noxdafox commented 4 years ago

Feature added in release v1.0.0.