No logs published to Cloudwatch

[ggrzesiuk]

Hi,

it seems there is an issue with publishing logs to cloudwatch. My cofiguration it as follows:

• RabbitMQ 3.9.8
• Erlang 24.1
• cloudwatch_exporter 1.0.1

my rabbitmq.conf

cloudwatch_exporter.aws.region = eu-west-1 cloudwatch_exporter.metrics.overview.enable = true cloudwatch_exporter.metrics.vhost.enable = true cloudwatch_exporter.metrics.node.enable = true cloudwatch_exporter.metrics.exchange.enable = true cloudwatch_exporter.metrics.queue.enable = true

advanced.config [{lager, [{handlers, [{lager_cloudwatch_backend, [info, "RabbitMQ"]}]}]}].

env variable set : AWS_DEFAULT_REGION=eu-west-1

Metrics are published correctly but there there are no logs in cloudwatch. I attached the following policy to my role to publish metrics and logs:

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "cloudwatch:", "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents", "logs:DescribeLogStreams" ], "Resource": [ "" ] } ] }

There are no errors in rabbitmq related to cloudwatch exporter plugin (even in debug mode). Any suggestions what might prevent logs publishing to cloudwatch ? Thanks, Greg

[noxdafox]

Where is the broker running? Is it on a AWS managed computing instance (EC2, ECS, ...) or not?

[ggrzesiuk]

It is running on AWS ( 3 EC2 instances).

[Sonic0]

@noxdafox @ggrzesiuk can I help with a test in my AWS account?

[ggrzesiuk]

@Sonic0 it would be great! I will be back in a week time. I could schedule a session for us to troubleshoot the issue on 3rd of November. Would that be OK for you ?

[noxdafox]

I tested locally and it was working. I don't have access to an AWS account to test it right now as my free-tier expired long ago.

Your policy looks fine from a first look. Did you try temporarily wildcarding (logs:*) the actions to ensure some action was not left behind?

[Sonic0]

Hi @ggrzesiuk. I think it'd be better if I wasn't. I'm not fluent in speaking English, so probably it's better that I just test the latest release and give you a feedback about eventual problems.

[Sonic0]

Hi @noxdafox @ggrzesiuk, I have created an EC2 with the latest versions of software:

• RabbitMQ 3.9.8
• Erlang 24.1.3

The EC2 has no problem publishing metrics to CloudWatch.

This is the IAM Policy associated with EC2 instance's AIM Role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "cloudwatch:PutMetricData"
            ],
            "Resource": "*"
        }
    ]
}

[noxdafox]

Thanks @Sonic0 but I believe the OP is having issues with logs and not metrics.

Metrics seem to work fine on his deployment too.

On my tests logs are being published as well.

[ggrzesiuk]

hi Guys,

correct. Metrics are published just fine. I tried already with different policies including logs:* but it did not help. @Sonic0 and/or @noxdafox would you be able to help me to troubleshoot on my AWS account ? Kindly please let me know your availibility( today?) so we could try to solve it. Thanks in advance, Greg

[Sonic0]

Hi @noxdafox @ggrzesiuk I'm sorry for the mistake. I have replicated the @ggrzesiuk configuration with the same result: There are no errors in rabbitmq related to cloudwatch exporter plugin (even in debug mode).

During this day I will make other tests about this problem.

[ggrzesiuk]

hi @Sonic0, have you managed to troubleshoot this issue ?

[Sonic0]

hi @Sonic0, have you managed to troubleshoot this issue ?

No, I haven't. I tried just an hour to troubleshoot this issue but nothing to do, no logs.

[ggrzesiuk]

@noxdafox and @Sonic0 would you mind if I schedule a session with you to try tro troubleshoot this issue? Maybe we are just missing something in configuration etc and it doesn't publish logs correctly. Your help would be very much appreciated. Tell whether today afternoon would work for you ? Anytime between 3 PM and 9 PM would work for me.

[ggrzesiuk]

@noxdafox, your help will be necessary here to troubleshoot this issue. Kindly please suggest suitable time to discuss or let me know whether I could provide some useful information on my setup/configuration.

[ggrzesiuk]

hi @noxdafox,

would you be able to investigate that issue and try to provide a fix ?

[hvillar]

hi @noxdafox, I have the same issue as @ggrzesiuk, is there some news about it ?