Allow configuration of dimensions on CloudWatchReporter

[djgeary]

It would be useful to have the configuration of the cloudwatch dimensions in the reporter. This would decouple the cloudwatch dimension concept from the metric name and not interfere with other reporters for the metric. This could be done via setting/injecting a dimension map on the reporter.

Taking the example in the documentation, the metric name would be left as "ServiceX Requests". The dimension map on machine A would contain the single entry machine=1.2.3.4 and on machine B would contain machine=7.5.3.1. The metric would then be reported to cloudwatch with the same dimensions as before, but other reporters would be unaware of the cloudwatch dimensions. For example JMX would just have "ServiceX Requests" as the metric name which is better in that case as you need to connect to each IP address directly anyway to get the JMX stats.

The existing functionality of determining the dimensions from the metric name could be left in place as an alternative if needed (OK if cloudwatch is the only reporter)

[santanusinha]

This would be very useful, as a lot of the dimensions would not change over the course of execution.

[djgeary]

After looking at this again it seems the idea of tags or dimensions may become a standard concept in metrics - see

http://metrics20.org/ https://groups.google.com/forum/#!topic/metrics-user/95Dv1xcOox0 https://github.com/dropwizard/metrics/pull/561 https://github.com/dropwizard/metrics/blob/796663609f310888240cc8afb58f75396f8391d2/metrics-core/src/main/java/io/dropwizard/metrics/MetricName.java

Also some dimensions that are not specific to the machine/environment eg business type dimensions instead of operational ones cannot be fixed in the reporter.

For these reasons it would make sense to keep the dimension definition in the metric name similar to what the CloudWatchReporter currently does, however the format would change to be something more standard that is understood by all reporters e.g.

ServiceX Requests environment=development machine=1.2.3.4 might become ServiceX Requests{environment=development,machine=1.2.3.4} or ServiceX Requests?environment=development,machine=1.2.3.4

or would make use of the MetricName concept under consideration for dropwizard metrics 4 which effectively includes the name and a map of tags. (This is now dependent on a future version of dropwizard metrics supporting this, and other reporters understanding the tags too)

However I think the concept of permutation / multiplicity should remain in the CloudWatchReporter as discussed previously ie move out of the metric name. As the documentation currently states ("CloudWatch does not aggregate metrics over dimensions on custom metrics") this is an issue with CloudWatch itself so the 'fix' for this should remain in the CloudWatchReporter. ie the concept of the * on dimensions should not be in the name itself. Instead (similar to the initial proposal above) a a Map<String, List<String>> of permutableDimensions could be injected in the CloudWatchReporter and these would be the ones used to create multiple metrics in CloudWatch

eg for a metric

ServiceX Requests{environment=development,machine=1.2.3.4}

if the permutableDimensions map contains

ServiceX Requests -> {machine}

then we create two CloudWatch submissions with and without the machine dimension

Other metrics systems eg graphite can handle the aggregations themselves so would not need this.

This is a better way to do it as really we want to say 'machine' is permutable - but currently we have to mark this in construction of the metric name in the application code and depending how this is done it might be possible to accidentally miss out on some combinations of metric name

It would also be useful to have a similar injected Map<String, List<String>> of ignorableDimensions. These could be dimensions that we want to ignore for CloudWatch (eg due to cost) but might want to see in other reporters.

We have implemented our own extension of CloudWatchReporter that includes these ideas.