Closed digitalkaoz closed 8 years ago
So in looking at the log snippets, I think the problem is with the ACL token... try setting the ACL token with CONSUL_ACL_TOKEN="" in your env vars to see if it makes the problem go away.
What version of Erlang are you running? The RabbitMQ startup banner in the logs can be helpful in finding the edge cases here.
I'll explore the bug side of that. It appears that the token query arg is both unset and added to the URI string incorrectly. That shouldn't happen. I'll have to look at why the tests wouldn't catch that.
I've confirmed the bug with not having a token set and am expanding test coverage for the consul module at the moment. I should have a 0.5.1 out soon that addresses the issue.
It appears the issue is appearing with all use of the token parameter so unfortunately there's no immediate workaround.
Would you mind giving this pre-release of 0.5.1 a quick test:
https://github.com/aweber/rabbitmq-autocluster/releases/tag/0.5.1-pr1
mh, i tested it. a bit better.
i still had to provide the ACL Environment Variable...startup works fine, but clustering not:
i see the rabbitmq
services created i my consul:
i use a single consul, so i need CONSUL_SVC_ADDR_AUTO: "true"
[{
"Node": {
"Node": "consul",
"Address": "172.17.0.7",
"TaggedAddresses": {
"wan": "172.17.0.7"
},
"CreateIndex": 3,
"ModifyIndex": 9
},
"Service": {
"ID": "rabbitmq:0a0dd6de799a",
"Service": "rabbitmq",
"Tags": null,
"Address": "0a0dd6de799a",
"Port": 5672,
"EnableTagOverride": false,
"CreateIndex": 6,
"ModifyIndex": 9
},
"Checks": [{
"Node": "consul",
"CheckID": "serfHealth",
"Name": "Serf Health Status",
"Status": "passing",
"Notes": "",
"Output": "Agent alive and reachable",
"ServiceID": "",
"ServiceName": "",
"CreateIndex": 3,
"ModifyIndex": 3
}, {
"Node": "consul",
"CheckID": "service:rabbitmq:0a0dd6de799a",
"Name": "Service 'rabbitmq' check",
"Status": "passing",
"Notes": "RabbitMQ Auto-Cluster Plugin TTL Check",
"Output": "",
"ServiceID": "rabbitmq:0a0dd6de799a",
"ServiceName": "rabbitmq",
"CreateIndex": 6,
"ModifyIndex": 9
}]
}, {
"Node": {
"Node": "consul",
"Address": "172.17.0.7",
"TaggedAddresses": {
"wan": "172.17.0.7"
},
"CreateIndex": 3,
"ModifyIndex": 9
},
"Service": {
"ID": "rabbitmq:90eb3ebedfac",
"Service": "rabbitmq",
"Tags": null,
"Address": "90eb3ebedfac",
"Port": 5672,
"EnableTagOverride": false,
"CreateIndex": 5,
"ModifyIndex": 8
},
"Checks": [{
"Node": "consul",
"CheckID": "serfHealth",
"Name": "Serf Health Status",
"Status": "passing",
"Notes": "",
"Output": "Agent alive and reachable",
"ServiceID": "",
"ServiceName": "",
"CreateIndex": 3,
"ModifyIndex": 3
}, {
"Node": "consul",
"CheckID": "service:rabbitmq:90eb3ebedfac",
"Name": "Service 'rabbitmq' check",
"Status": "passing",
"Notes": "RabbitMQ Auto-Cluster Plugin TTL Check",
"Output": "",
"ServiceID": "rabbitmq:90eb3ebedfac",
"ServiceName": "rabbitmq",
"CreateIndex": 5,
"ModifyIndex": 8
}]
}]
the container ips of my rabbits are:
i guess the Node
Section is wrong? shouldnt there be the rabbitmq node ip?
i see both nodes, but still the cluster wont be formed :/
The full startup logs would be helpful there. If everything got that far, my guess is one of two items are biting you:
.erlang.cookie
file (the startup would confirm if this is the case)0a0dd6de799a
is not able to resolve container 90eb3ebedfac
im using docker on my local machine, so no firewalling!
the startup log:
$ docker logs 0a0dd6de799a
=ERROR REPORT==== 2-Jun-2016::21:05:25 ===
Failed to retrieve AWS credentials: undefined
=INFO REPORT==== 2-Jun-2016::21:05:25 ===
Starting RabbitMQ 3.6.2 on Erlang 18.3
Copyright (C) 2007-2016 Pivotal Software, Inc.
Licensed under the MPL. See http://www.rabbitmq.com/
RabbitMQ 3.6.2. Copyright (C) 2007-2016 Pivotal Software, Inc.
## ## Licensed under the MPL. See http://www.rabbitmq.com/
## ##
########## Logs: tty
###### ## tty
##########
Starting broker...
=INFO REPORT==== 2-Jun-2016::21:05:25 ===
node : rabbit@0a0dd6de799a
home dir : /var/lib/rabbitmq
config file(s) : /etc/rabbitmq/rabbitmq.config
cookie hash : avIdrkSVmb3GWEFtBz8hvA==
log : tty
sasl log : tty
database dir : /var/lib/rabbitmq/mnesia/rabbit@0a0dd6de799a
=INFO REPORT==== 2-Jun-2016::21:05:28 ===
autocluster: Delaying startup for 4223ms.
=INFO REPORT==== 2-Jun-2016::21:05:32 ===
autocluster: Starting consul registration.
=INFO REPORT==== 2-Jun-2016::21:05:32 ===
autocluster: Registering node with consul.
=INFO REPORT==== 2-Jun-2016::21:05:32 ===
Memory limit set to 1579MB of 3949MB total.
=INFO REPORT==== 2-Jun-2016::21:05:32 ===
Disk free limit set to 50MB
=INFO REPORT==== 2-Jun-2016::21:05:32 ===
Limiting to approx 1048476 file handles (943626 sockets)
=INFO REPORT==== 2-Jun-2016::21:05:32 ===
FHC read buffering: OFF
FHC write buffering: ON
=INFO REPORT==== 2-Jun-2016::21:05:32 ===
Database directory at /var/lib/rabbitmq/mnesia/rabbit@0a0dd6de799a is empty. Initialising from scratch...
=INFO REPORT==== 2-Jun-2016::21:05:32 ===
application: mnesia
exited: stopped
type: temporary
=INFO REPORT==== 2-Jun-2016::21:05:32 ===
Priority queues enabled, real BQ is rabbit_variable_queue
=INFO REPORT==== 2-Jun-2016::21:05:32 ===
Management plugin: using rates mode 'basic'
=INFO REPORT==== 2-Jun-2016::21:05:32 ===
msg_store_transient: using rabbit_msg_store_ets_index to provide index
=INFO REPORT==== 2-Jun-2016::21:05:32 ===
msg_store_persistent: using rabbit_msg_store_ets_index to provide index
=WARNING REPORT==== 2-Jun-2016::21:05:32 ===
msg_store_persistent: rebuilding indices from scratch
=INFO REPORT==== 2-Jun-2016::21:05:32 ===
Adding vhost '/'
=INFO REPORT==== 2-Jun-2016::21:05:32 ===
Creating user 'guest'
=INFO REPORT==== 2-Jun-2016::21:05:32 ===
Setting user tags for user 'guest' to [administrator]
=INFO REPORT==== 2-Jun-2016::21:05:32 ===
Setting permissions for 'guest' in '/' to '.*', '.*', '.*'
=INFO REPORT==== 2-Jun-2016::21:05:32 ===
started TCP Listener on [::]:5672
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Management plugin started. Port: 15672
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics event collector started.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics channel stats collector started.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics queue stats collector started.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics database started.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_queue_stats_fine_stats,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_queue_stats_deliver_get,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_queue_stats_queue_msg_counts,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_queue_stats_queue_msg_rates,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_queue_exchange_stats_fine_stats,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_vhost_stats_deliver_get,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_vhost_stats_fine_stats,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_vhost_stats_queue_msg_rates,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_vhost_stats_queue_msg_counts,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_vhost_stats_coarse_conn_stats,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_channel_queue_stats_deliver_get,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_channel_queue_stats_fine_stats,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_channel_queue_stats_queue_msg_counts,
5000}.
completed with 8 plugins.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Server startup complete; 8 plugins started.
* rabbitmq_management
* rabbitmq_web_dispatch
* webmachine
* mochiweb
* rabbitmq_management_agent
* amqp_client
* autocluster
* rabbitmq_aws
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_channel_stats_deliver_get,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_channel_stats_fine_stats,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_channel_stats_queue_msg_counts,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_channel_exchange_stats_deliver_get,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_channel_exchange_stats_fine_stats,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_exchange_stats_fine_stats,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_node_stats_coarse_node_stats,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_node_node_stats_coarse_node_node_stats,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_connection_stats_coarse_conn_stats,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {channel_stats,5000}.
=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {connection_stats,5000}.
and
$ docker logs 90eb3ebedfac
=ERROR REPORT==== 2-Jun-2016::21:05:25 ===
Failed to retrieve AWS credentials: undefined
=INFO REPORT==== 2-Jun-2016::21:05:25 ===
Starting RabbitMQ 3.6.2 on Erlang 18.3
Copyright (C) 2007-2016 Pivotal Software, Inc.
Licensed under the MPL. See http://www.rabbitmq.com/
RabbitMQ 3.6.2. Copyright (C) 2007-2016 Pivotal Software, Inc.
## ## Licensed under the MPL. See http://www.rabbitmq.com/
## ##
########## Logs: tty
###### ## tty
##########
Starting broker...
=INFO REPORT==== 2-Jun-2016::21:05:25 ===
node : rabbit@90eb3ebedfac
home dir : /var/lib/rabbitmq
config file(s) : /etc/rabbitmq/rabbitmq.config
cookie hash : avIdrkSVmb3GWEFtBz8hvA==
log : tty
sasl log : tty
database dir : /var/lib/rabbitmq/mnesia/rabbit@90eb3ebedfac
=INFO REPORT==== 2-Jun-2016::21:05:28 ===
autocluster: Delaying startup for 2649ms.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
autocluster: Starting consul registration.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
autocluster: Registering node with consul.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Memory limit set to 1579MB of 3949MB total.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Disk free limit set to 50MB
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Limiting to approx 1048476 file handles (943626 sockets)
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
FHC read buffering: OFF
FHC write buffering: ON
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Database directory at /var/lib/rabbitmq/mnesia/rabbit@90eb3ebedfac is empty. Initialising from scratch...
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
application: mnesia
exited: stopped
type: temporary
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Priority queues enabled, real BQ is rabbit_variable_queue
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Management plugin: using rates mode 'basic'
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
msg_store_transient: using rabbit_msg_store_ets_index to provide index
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
msg_store_persistent: using rabbit_msg_store_ets_index to provide index
=WARNING REPORT==== 2-Jun-2016::21:05:31 ===
msg_store_persistent: rebuilding indices from scratch
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Adding vhost '/'
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Creating user 'guest'
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Setting user tags for user 'guest' to [administrator]
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Setting permissions for 'guest' in '/' to '.*', '.*', '.*'
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
started TCP Listener on [::]:5672
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Management plugin started. Port: 15672
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics event collector started.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics channel stats collector started.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics queue stats collector started.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics database started.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_queue_stats_fine_stats,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_queue_stats_deliver_get,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_queue_stats_queue_msg_counts,
5000}.
completed with 8 plugins.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_queue_stats_queue_msg_rates,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Server startup complete; 8 plugins started.
* rabbitmq_management
* rabbitmq_web_dispatch
* webmachine
* mochiweb
* rabbitmq_management_agent
* amqp_client
* autocluster
* rabbitmq_aws
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_queue_exchange_stats_fine_stats,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_vhost_stats_deliver_get,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_vhost_stats_fine_stats,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_vhost_stats_queue_msg_rates,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_vhost_stats_queue_msg_counts,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_vhost_stats_coarse_conn_stats,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_channel_queue_stats_deliver_get,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_channel_queue_stats_fine_stats,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_channel_queue_stats_queue_msg_counts,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_channel_stats_deliver_get,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_channel_stats_fine_stats,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_channel_stats_queue_msg_counts,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_channel_exchange_stats_deliver_get,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_channel_exchange_stats_fine_stats,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_exchange_stats_fine_stats,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_node_stats_coarse_node_stats,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_node_node_stats_coarse_node_node_stats,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_connection_stats_coarse_conn_stats,
5000}.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {channel_stats,5000}.
=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {connection_stats,5000}.
both nodes can reach each other:
root@0a0dd6de799a:/# dig rabbitmq.service.consul
; <<>> DiG 9.9.5-9+deb8u6-Debian <<>> rabbitmq.service.consul
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 13360
;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available
;; QUESTION SECTION:
;rabbitmq.service.consul. IN A
;; ANSWER SECTION:
rabbitmq.service.consul. 0 IN CNAME 0a0dd6de799a.
rabbitmq.service.consul. 0 IN CNAME 90eb3ebedfac.
but: they are not accessible by nodename, just by ip:
root@0a0dd6de799a:/# telnet 90eb3ebedfac 5672
telnet: could not resolve 90eb3ebedfac/5672: Name or service not known
root@0a0dd6de799a:/# telnet 172.17.0.8 5672
Trying 172.17.0.8...
Connected to 172.17.0.8.
Escape character is '^]'.
^C^CConnection closed by foreign host.
root@0a0dd6de799a:/# telnet 172.17.0.9 5672
Trying 172.17.0.9...
Connected to 172.17.0.9.
Escape character is '^]'
so i guess it could be fixed by setting the ip in consul instead of the hostname?
Cookies match, which is good :)
What about dig 90eb3ebedfac
on that container?
Also, adding:
{log_levels, [{autocluster, debug}]}
to your RabbitMQ config could help straighten out what's up. Are you using the aweber/rabbitmq-autocluster
image, or your own? I should make a way to turn debugging logs on easier.
Hmm what about 90eb3ebedfac.rabbitmq.service.consul
?
nope that wont work, i guess
CONSUL_SVC_ADDR_AUTO
should be able to propagate the ip instead of the hostname, but how to get the right ip?
as im using docker, i dont know which ip the container gets
CONSUL_SVC_ADDR
would be right if i only knew the ip on startup so i can set this env var
hostname --ip-address
would be right i think!
maybe you can add an environment variable which decides if CONSUL_SVC_ADDR_AUTO
exposes the hostname or the ip, maybe CONSUL_SVC_ADDR_IP=true|false
But that might be problematic in overlay networks with multiple public ips. Best would be elasticsearch s solution: https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html#network-interface-values define an network interface to get the ip from....
Sadly im not familiar with erlang, otherwise i would help :)
Gavin M. Roy notifications@github.com schrieb am Do., 2. Juni 2016, 23:27:
Hmm what about 90eb3ebedfac.rabbitmq.service.consul?
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/aweber/rabbitmq-autocluster/issues/60#issuecomment-223428047, or mute the thread https://github.com/notifications/unsubscribe/AAR615e7sL5c1h68GerbLo7_Zl01kV1kks5qH0rVgaJpZM4IsXpv .
Hmm ok, that sounds reasonable -- I was debating over that when I wrote the code for the ADDR_AUTO; unfortunately I scrapped my code that parsed the data structure that returns it. The first version of ADDR_AUTO used IP address.
Maybe add it back and let the User decide? :)
Gavin M. Roy notifications@github.com schrieb am Fr., 3. Juni 2016, 00:27:
Hmm ok, that sounds reasonable -- I was debating over that when I wrote the code for the ADDR_AUTO; unfortunately I scrapped my code that parsed the data structure that returns it. The first version of ADDR_AUTO used IP address.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/aweber/rabbitmq-autocluster/issues/60#issuecomment-223441313, or mute the thread https://github.com/notifications/unsubscribe/AAR617mfEzp6ewetoo6iw8IWb1rsCdFiks5qH1jHgaJpZM4IsXpv .
I've added CONSUL_SVC_ADDR_NIC
for 0.6.0 and am gearing up to release that now.
0.6.0 has been released. I'll close this issue out, feel free to re-open if you continue to have an issue.
You are awesome Dude! Will test it later today and give you some Feedback :)
Gavin M. Roy notifications@github.com schrieb am Sa., 4. Juni 2016, 01:07:
Closed #60 https://github.com/aweber/rabbitmq-autocluster/issues/60.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/aweber/rabbitmq-autocluster/issues/60#event-681755853, or mute the thread https://github.com/notifications/unsubscribe/AAR61yHbeR2_HE6lwLEngGBpwXPwcKxFks5qILPMgaJpZM4IsXpv .
hey, thanks for your work!
im having trouble getting a consul discovery working:
logs: (not useful :( )
ENV Vars:
the consul is accessable:
also i dont see services generated in the consul, as i understood the workflow, each node should register itself under the service name
rabbitmq
but i dont see them. im using the https://github.com/gliderlabs/registrator and already have all my rabbitmqs registered asrabbitmq-5672
so a
dig rabbitmq-5672.service.consul
would give me all ips of the rabbitmq nodes...what am i doing wrong?