cloudwatch exporter - exporting logs issue

[ggrzesiuk]

Hi,

I have an issue when using cloudwatch exporter plugin. My Rabbitmq cluster consist of 3 nodes running on AWS EC2 instances. I'm using IAM role with attached policy to grant access to logs and cloudwatch.

I want to use cloudwatch exporter plugin to export logs and metrics. Unfortunately it seems that only metrics are expoerted but not logs. I'm getting the following errors in the logs: error] <0.354.0> Lager failed to install handler lager_cloudwatch_backend into rabbit_log_upgrade_lager_event, retrying later : {'EXIT', {undef, [{crypto, hmac, [sha256, "AWS4NEh8lTR6nL32s1qzS++EpOZ6A/veQlpVIKjXgrYk", "20210927"], []}, {erlcloud_aws, signing_key, 4, [{file, "src/erlcloud_aws.erl"}, {line, 1128}]}, {erlcloud_aws, sign_v4, 8, [{file, "src/erlcloud_aws.erl"}, {line, 1048}]}, {erlcloud_cloudwatch_logs, make_request_headers, 3, [{file, "src/erlcloud_cloudwatch_logs.erl"}, {line, 703}]}, {erlcloud_cloudwatch_logs, maybe_cw_request, 3, [{file, "src/erlcloud_cloudwatch_logs.erl"}, {line, 687}]}, {erlcloud_cloudwatch_logs, describe_log_groups, 4, [{file, "src/erlcloud_cloudwatch_logs.erl"}, {line, 334}]}, {lager_cloudwatch_backend, maybe_create_log_group, 1, [{file, "src/lager_cloudwatch_backend.erl"}, {line, 129}]}, {lager_cloudwatch_backend, init, 1, [{file, "src/lager_cloudwatch_backend.erl"}, {line, 78}]}]}}

My rabbitmq.conf setting related to plugin:

cloudwatch_exporter.aws.region = eu-west-1 cloudwatch_exporter.metrics.overview.enable = true cloudwatch_exporter.metrics.vhost.enable = true cloudwatch_exporter.metrics.node.enable = true cloudwatch_exporter.metrics.exchange.enable = true cloudwatch_exporter.metrics.queue.enable = true log.file.level = debug

and in advanced.conf [{lager, [{handlers, [{lager_cloudwatch_backend, [debug, "RabbitMQ", "rabbitmq-sandbox"]}]}]}].

Any advice with this issue ?

[noxdafox]

This looks like issue #34. What version of RabbitMQ and Erlang are you using?

I am assuming you are using Erlang 24, does the problem persist when downgrading to 23?

[ggrzesiuk]

Hi,

yes I was using the lastest rabbitmq and erlang version and was getting this issue. Now I downgraded RabbitMQ 3.8.23 and Erlang 23.3.1 but get different error. I added information about it in [https://github.com/noxdafox/rabbitmq-cloudwatch-exporter/issues/31] It looks very similar like issue 31.

[ggrzesiuk]

hi @noxdafox , do you have any suggestions for a work around of this issue ?

Thanks, Greg

[noxdafox]

Sorry for the late reply, I am very busy in this period.

I will try to make a new release of lager_cloudwatch to fix this issue over this WE.

[noxdafox]

Issue resolved in release v1.0.1. Please re-open this ticket if the problem persists.