RabbitMQ errors in log file

[rjbaucells]

Hi,

I just managed to configure the rabbitmq-cloudwatch-exporter plugin in my RabbitMQ cluster and it is sending metrics to CloudWatch but I am getting the following error messages into the log file at a regular interval (~2mins).

This is my configuration:

listeners.tcp.default = 5672
management.tcp.port = 15672
proxy_protocol = false
loopback_users.guest = false

vm_memory_high_watermark.relative = 0.850
default_vhost = dev
default_user = rabbitmq-user
default_pass = my_secure_password

cluster_formation.peer_discovery_backend = rabbit_peer_discovery_aws
cluster_formation.aws.region = us-east-1
cluster_formation.aws.use_autoscaling_group = true

log.console = true
log.console.level = info
log.default.level = info

cloudwatch_exporter.aws.region = "us-east-1"
cloudwatch_exporter.metrics.1 = overview
cloudwatch_exporter.metrics.2 = vhost
cloudwatch_exporter.metrics.3 = node
cloudwatch_exporter.metrics.4 = exchange
cloudwatch_exporter.metrics.5 = queue
cloudwatch_exporter.metrics.6 = connection
cloudwatch_exporter.metrics.7 = channel

The following error messages are triggering our log file alarms:

2019-10-19 03:00:58.888 [error] <0.24899.20> ** Generic server rabbitmq_cloudwatch_exporter terminating
** Last message in was export_metrics
** When Server state == [[{period,60},{collectors,[overview,vhost,node,exchange,queue,connection,channel]},{namespace,<<"RabbitMQ">>}],[{region,<<"us-east-1">>}]]
** Reason for termination ==
** {#{'__exception__' => true,'__struct__' => 'Elixir.ExAws.Error',message => <<"ExAws Request Error!\n\n{:error, {:http_error, 400, %{body: \"<ErrorResponse xmlns=\\\"http://monitoring.amazonaws.com/doc/2010-08-01/\\\">\\n <Error>\\n <Type>Sender</Type>\\n <Code>MissingParameter</Code>\\n <Message>The parameter MetricData.member.17.Dimensions.member.3.Value is required.\\nThe parameter MetricData.member.18.Dimensions.member.3.Value is required.</Message>\\n </Error>\\n <RequestId>49c3c87b-87d0-4247-9507-b11b6d063b4a</RequestId>\\n</ErrorResponse>\\n\", headers: [{\"x-amzn-RequestId\", \"49c3c87b-87d0-4247-9507-b11b6d063b4a\"}, {\"Content-Type\", \"text/xml\"}, {\"Content-Length\", \"401\"}, {\"Date\", \"Sat, 19 Oct 2019 03:00:58 GMT\"}, {\"Connection\", \"close\"}], status_code: 400}}}\n">>},[{'Elixir.ExAws','request!',2,[{file,"lib/ex_aws.ex"},{line,66}]},{'Elixir.Enum','-map/2-lists^map/1-0-',2,[{file,"lib/enum.ex"},{line,1327}]},{'Elixir.Enum','-map/2-lists^map/1-0-',2,[{file,"lib/enum.ex"},{line,1327}]},{'Elixir.RabbitMQCloudWatchExporter.Exporter',handle_info,2,[{file,"lib/rabbitmq_cloudwatch_exporter/exporter.ex"},{line,60}]},{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,637}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,711}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}
2019-10-19 03:00:58.889 [error] <0.24899.20> CRASH REPORT Process <0.24899.20> with 1 neighbours crashed with reason: #{'__exception__' => true,'__struct__' => 'Elixir.ExAws.Error',message => <<"ExAws Request Error!\n\n{:error, {:http_error, 400, %{body: \"<ErrorResponse xmlns=\\\"http://monitoring.amazonaws.com/doc/2010-08-01/\\\">\\n <Error>\\n <Type>Sender</Type>\\n <Code>MissingParameter</Code>\\n <Message>The parameter MetricData.member.17.Dimensions.member.3.Value is required.\\nThe parameter MetricData.member.18.Dimensions.member.3.Value is required.</Message>\\n </Error>\\n <RequestId>49c3c87b-87d0-424...">>} in 'Elixir.ExAws':'request!'/2 line 66
2019-10-19 03:00:58.891 [error] <0.189.0> Supervisor 'Elixir.Singleton.Supervisor' had child 'Elixir.Singleton.Manager' started with 'Elixir.Singleton.Manager':start_link('Elixir.RabbitMQCloudWatchExporter.Exporter', [], rabbitmq_cloudwatch_exporter, 'singleton_0bpF3SXqPZTQc80tAf+lu7E5kqQ') at <0.23548.20> exit with reason #{'__exception__' => true,'__struct__' => 'Elixir.ExAws.Error',message => <<"ExAws Request Error!\n\n{:error, {:http_error, 400, %{body: \"<ErrorResponse xmlns=\\\"http://monitoring.amazonaws.com/doc/2010-08-01/\\\">\\n <Error>\\n <Type>Sender</Type>\\n <Code>MissingParameter</Code>\\n <Message>The parameter MetricData.member.17.Dimensions.member.3.Value is required.\\nThe parameter MetricData.member.18.Dimensions.member.3.Value is required.</Message>\\n </Error>\\n <RequestId>49c3c87b-87d0-424...">>} in 'Elixir.ExAws':'request!'/2 line 66 in context child_terminated

[oytuntez]

We receive very similar messages:

** {#{'__exception__' => true,'__struct__' => 'Elixir.ExAws.Error',message => <<"ExAws Request Error!\n\n{:error, {:http_error, 400, %{body: \"<ErrorResponse xmlns=\\\"http://monitoring.amazonaws.com/doc/2010-08-01/\\\">\\n <Error>\\n <Type>Sender</Type>\\n <Code>MissingParameter</Code>\\n <Message>The parameter MetricData.member.5.Dimensions.member.3.Value is required.\\nThe parameter MetricData.member.6.Dimensions.member.3.Value is required.</Message>\\n </Error>\\n <RequestId>d3dfb2eb-18cc-4c5f-9e97-567743f27e8d</RequestId>\\n</ErrorResponse>\\n\", headers: [{\"x-amzn-RequestId\", \"d3dfb2eb-18cc-4c5f-9e97-567743f27e8d\"}, {\"Content-Type\", \"text/xml\"}, {\"Content-Length\", \"399\"}, {\"Date\", \"Wed, 30 Oct 2019 16:17:54 GMT\"}, {\"Connection\", \"close\"}], status_code: 400}}}\n">>},[{'Elixir.ExAws','request!',2,[{file,"lib/ex_aws.ex"},{line,66}]},{'Elixir.Enum','-map/2-lists^map/1-0-',2,[{file,"lib/enum.ex"},{line,1327}]},{'Elixir.Enum','-map/2-lists^map/1-0-',2,[{file,"lib/enum.ex"},{line,1327}]},{'Elixir.RabbitMQCloudWatchExporter.Exporter',handle_info,2,[{file,"lib/rabbitmq_cloudwatch_exporter/exporter.ex"},{line,60}]},{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,637}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,711}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,249}]}]}

[noxdafox]

This ticket is a duplicated of #7.

Please follow the instructions in the above ticket to help debugging this issue.

Closing this as duplicate.