noxdafox / rabbitmq-cloudwatch-exporter

RabbitMQ Plugin for publishing cluster metrics to AWS CloudWatch
Mozilla Public License 2.0
41 stars 9 forks source link

Lager failed to install handler lager_cloudwatch_backend into lager_event #31

Open an2ny opened 3 years ago

an2ny commented 3 years ago

trying to set this up with rabbitmq but errors.

here's my advanced.config

[{lager,
  [{handlers, [{lager_cloudwatch_backend, [debug, "RabbitMQ", "stream1"]}]}]}].

rabbitmq.confg

cloudwatch_exporter.aws.access_key_id="asdsadsa"
cloudwatch_exporter.aws.secret_access_key="asdsadsaada"
cloudwatch_exporter.aws.region="us-east-2"
cloudwatch_exporter.metrics.overview.enable = true
cloudwatch_exporter.metrics.vhost.enable = true
cloudwatch_exporter.metrics.node.enable = true
cloudwatch_exporter.metrics.exchange.enable = true
cloudwatch_exporter.metrics.queue.enable = true

log.file.level = debug
log.console = true
log.console.level = debug
log.connection.level = debug
log.channel.level = debug
loopback_users = none

and the error:

2021-03-07 15:56:48.294 [error] <0.289.0> Lager failed to install handler lager_cloudwatch_backend into lager_event, retrying later : {'EXIT',

{{case_clause,

{error,

{socket_error,

{econnrefused,

[{lhttpc_client,

send_request,

1,

[{file,

"src/lhttpc_client.erl"},

{line,

252}]},

{lhttpc_client,

execute,

9,

[{file,

"src/lhttpc_client.erl"},

{line,

201}]},

{lhttpc_client,

request,

9,

[{file,

"src/lhttpc_client.erl"},

{line,

101}]}]}}}},

[{lager_cloudwatch_backend,

maybe_create_log_group,

1,

[{file,

"src/lager_cloudwatch_backend.erl"},

{line,

129}]},

{lager_cloudwatch_backend,

init,

1,

[{file,

"src/lager_cloudwatch_backend.erl"},

{line,

78}]},

{gen_event,

server_add_handler,

4,

[{file,

"gen_event.erl"},

{line,

516}]},

{gen_event,

handle_msg,

6,

[{file,

"gen_event.erl"},

{line,

361}]},

{proc_lib,

init_p_do_apply,

3,

[{file,

"proc_lib.erl"},

{line,

226}]}]}}

i hope someone can help. thanks

noxdafox commented 3 years ago

This line,

{socket_error, {econnrefused, ...} ...}

suggests that your instance cannot reach AWS CloudWatch service. Check the network settings and your firewalls to make sure the instance can reach out AWS services.

an2ny commented 3 years ago

@noxdafox do you have any idea what ports those might be? i am doing this using docker so it could be the reason

noxdafox commented 3 years ago

You can refer to AWS documentation to know what are the requirements for your environment to work.

https://docs.aws.amazon.com/workspaces/latest/adminguide/workspaces-port-requirements.html

noxdafox commented 3 years ago

Is this issue still ongoing?

an2ny commented 3 years ago

Is this issue still ongoing?

i have given up on this. i opened all ports mentioned in https://docs.aws.amazon.com/workspaces/latest/adminguide/workspaces-port-requirements.html in docker and still no luck.

GrigoriLab commented 3 years ago

Is this issue still ongoing?

i have given up on this. i opened all ports mentioned in https://docs.aws.amazon.com/workspaces/latest/adminguide/workspaces-port-requirements.html in docker and still no luck.

I had the same problem. It turned out if you are testing on local machine just having .aws in your home folder (for rabbitmq docker image it should be in /var/lib/rabbitmq/.aws) is not working, you should have explicitly set aws environment variables:

root@01382c1e6bc6:/# export AWS_ACCESS_KEY_ID=<aws_access_key_id>
root@01382c1e6bc6:/# export AWS_SECRET_ACCESS_KEY=<aws_secret_access_key>
root@01382c1e6bc6:/# export AWS_DEFAULT_REGION=<region>
root@01382c1e6bc6:/# rabbitmq-server
ggrzesiuk commented 2 years ago

Hi I have a similar issue. I'm getting the following error. Lager failed to install handler lager_cloudwatch_backend into lager_event, retrying later : {'EXIT', {{case_clause, {error, {socket_error, timeout}}}, [{lager_cloudwatch_backend, maybe_create_log_group, 1, [{file, "src/lager_cloudwatch_backend.erl"}, {line, 129}]}, {lager_cloudwatch_backend, init, 1, [{file, "src/lager_cloudwatch_backend.erl"}, {line, 78}]}, {gen_event, server_add_handler, 4, [{file, "gen_event.erl"}, {line, 516}]}, {gen_event, handle_msg, 6, [{file, "gen_event.erl"}, {line, 361}]}, {proc_lib, init_p_do_apply, 3, [{file, "proc_lib.erl"}, {line, 226}]}]}}

I'm using RabbitMQ 3.8.23 and Erlang 23.3.1

ggrzesiuk commented 2 years ago

Any other sugestions why logs are not beeing published to cloudwatch and only metrics ?