aweber / rabbitmq-autocluster

This project is now maintained by the RabbitMQ Team, visit the official repo @
https://github.com/rabbitmq/rabbitmq-autocluster
BSD 3-Clause "New" or "Revised" License
336 stars 120 forks source link

cant get consul discovery working #60

Closed digitalkaoz closed 8 years ago

digitalkaoz commented 8 years ago

hey, thanks for your work!

im having trouble getting a consul discovery working:

logs: (not useful :( )

Error description:
   {could_not_start,rabbit,
       {badarg,
           [{re,run,
                [[104,116,116,112,58,47,47,99,111,110,115,117,108,46,115,101,
                  114,118,105,99,101,46,99,111,110,115,117,108,58,56,53,48,48,
                  47,118,49,47,104,101,97,108,116,104,47,115,101,114,118,105,
                  99,101,47,114,97,98,98,105,116,109,113,63,112,97,115,115,
                  105,110,103,38,
                  {token,false}],
                 ":",
                 [{capture,first}]],
                []},
            {http_uri,split_uri,5,[{file,"http_uri.erl"},{line,199}]},
            {http_uri,parse_scheme,2,[{file,"http_uri.erl"},{line,137}]},
            {http_uri,parse,2,[{file,"http_uri.erl"},{line,89}]},
            {httpc,request,5,[{file,"httpc.erl"},{line,164}]},
            {autocluster_httpc,get,5,
                [{file,"src/autocluster_httpc.erl"},{line,106}]},
            {autocluster_consul,nodelist,0,
                [{file,"src/autocluster_consul.erl"},{line,55}]},
            {autocluster,ensure_registered,2,
                [{file,"src/autocluster.erl"},{line,83}]}]}}

ENV Vars:

                "AUTOCLUSTER_TYPE=consul",
                "CONSUL_HOST=consul.service.consul",
                "CONSUL_PORT=8500",
                "RABBITMQ_VERSION=3.6.2"

the consul is accessable:

$ docker exec a78e625a498f sh -c "wget consul.service.consul:8500"
converted 'http://consul.service.consul:8500' (ANSI_X3.4-1968) -> 'http://consul.service.consul:8500' (UTF-8)
--2016-06-02 09:30:07--  http://consul.service.consul:8500/
Resolving consul.service.consul (consul.service.consul)... 172.17.0.5
Connecting to consul.service.consul (consul.service.consul)|172.17.0.5|:8500... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: /ui/ [following]
converted 'http://consul.service.consul:8500/ui/' (ANSI_X3.4-1968) -> 'http://consul.service.consul:8500/ui/' (UTF-8)
--2016-06-02 09:30:07--  http://consul.service.consul:8500/ui/
Reusing existing connection to consul.service.consul:8500.
HTTP request sent, awaiting response... 200 OK

also i dont see services generated in the consul, as i understood the workflow, each node should register itself under the service name rabbitmq but i dont see them. im using the https://github.com/gliderlabs/registrator and already have all my rabbitmqs registered as rabbitmq-5672

so a dig rabbitmq-5672.service.consul would give me all ips of the rabbitmq nodes...

what am i doing wrong?

gmr commented 8 years ago

So in looking at the log snippets, I think the problem is with the ACL token... try setting the ACL token with CONSUL_ACL_TOKEN="" in your env vars to see if it makes the problem go away.

What version of Erlang are you running? The RabbitMQ startup banner in the logs can be helpful in finding the edge cases here.

I'll explore the bug side of that. It appears that the token query arg is both unset and added to the URI string incorrectly. That shouldn't happen. I'll have to look at why the tests wouldn't catch that.

gmr commented 8 years ago

I've confirmed the bug with not having a token set and am expanding test coverage for the consul module at the moment. I should have a 0.5.1 out soon that addresses the issue.

It appears the issue is appearing with all use of the token parameter so unfortunately there's no immediate workaround.

gmr commented 8 years ago

Would you mind giving this pre-release of 0.5.1 a quick test:

https://github.com/aweber/rabbitmq-autocluster/releases/tag/0.5.1-pr1

digitalkaoz commented 8 years ago

mh, i tested it. a bit better.

i still had to provide the ACL Environment Variable...startup works fine, but clustering not:

i see the rabbitmq services created i my consul:

i use a single consul, so i need CONSUL_SVC_ADDR_AUTO: "true"

[{
    "Node": {
        "Node": "consul",
        "Address": "172.17.0.7",
        "TaggedAddresses": {
            "wan": "172.17.0.7"
        },
        "CreateIndex": 3,
        "ModifyIndex": 9
    },
    "Service": {
        "ID": "rabbitmq:0a0dd6de799a",
        "Service": "rabbitmq",
        "Tags": null,
        "Address": "0a0dd6de799a",
        "Port": 5672,
        "EnableTagOverride": false,
        "CreateIndex": 6,
        "ModifyIndex": 9
    },
    "Checks": [{
        "Node": "consul",
        "CheckID": "serfHealth",
        "Name": "Serf Health Status",
        "Status": "passing",
        "Notes": "",
        "Output": "Agent alive and reachable",
        "ServiceID": "",
        "ServiceName": "",
        "CreateIndex": 3,
        "ModifyIndex": 3
    }, {
        "Node": "consul",
        "CheckID": "service:rabbitmq:0a0dd6de799a",
        "Name": "Service 'rabbitmq' check",
        "Status": "passing",
        "Notes": "RabbitMQ Auto-Cluster Plugin TTL Check",
        "Output": "",
        "ServiceID": "rabbitmq:0a0dd6de799a",
        "ServiceName": "rabbitmq",
        "CreateIndex": 6,
        "ModifyIndex": 9
    }]
}, {
    "Node": {
        "Node": "consul",
        "Address": "172.17.0.7",
        "TaggedAddresses": {
            "wan": "172.17.0.7"
        },
        "CreateIndex": 3,
        "ModifyIndex": 9
    },
    "Service": {
        "ID": "rabbitmq:90eb3ebedfac",
        "Service": "rabbitmq",
        "Tags": null,
        "Address": "90eb3ebedfac",
        "Port": 5672,
        "EnableTagOverride": false,
        "CreateIndex": 5,
        "ModifyIndex": 8
    },
    "Checks": [{
        "Node": "consul",
        "CheckID": "serfHealth",
        "Name": "Serf Health Status",
        "Status": "passing",
        "Notes": "",
        "Output": "Agent alive and reachable",
        "ServiceID": "",
        "ServiceName": "",
        "CreateIndex": 3,
        "ModifyIndex": 3
    }, {
        "Node": "consul",
        "CheckID": "service:rabbitmq:90eb3ebedfac",
        "Name": "Service 'rabbitmq' check",
        "Status": "passing",
        "Notes": "RabbitMQ Auto-Cluster Plugin TTL Check",
        "Output": "",
        "ServiceID": "rabbitmq:90eb3ebedfac",
        "ServiceName": "rabbitmq",
        "CreateIndex": 5,
        "ModifyIndex": 8
    }]
}]

the container ips of my rabbits are:

i guess the Node Section is wrong? shouldnt there be the rabbitmq node ip? i see both nodes, but still the cluster wont be formed :/

gmr commented 8 years ago

The full startup logs would be helpful there. If everything got that far, my guess is one of two items are biting you:

digitalkaoz commented 8 years ago

im using docker on my local machine, so no firewalling!

the startup log:

$ docker logs 0a0dd6de799a

=ERROR REPORT==== 2-Jun-2016::21:05:25 ===
Failed to retrieve AWS credentials: undefined

=INFO REPORT==== 2-Jun-2016::21:05:25 ===
Starting RabbitMQ 3.6.2 on Erlang 18.3
Copyright (C) 2007-2016 Pivotal Software, Inc.
Licensed under the MPL.  See http://www.rabbitmq.com/

              RabbitMQ 3.6.2. Copyright (C) 2007-2016 Pivotal Software, Inc.
  ##  ##      Licensed under the MPL.  See http://www.rabbitmq.com/
  ##  ##
  ##########  Logs: tty
  ######  ##        tty
  ##########
              Starting broker...

=INFO REPORT==== 2-Jun-2016::21:05:25 ===
node           : rabbit@0a0dd6de799a
home dir       : /var/lib/rabbitmq
config file(s) : /etc/rabbitmq/rabbitmq.config
cookie hash    : avIdrkSVmb3GWEFtBz8hvA==
log            : tty
sasl log       : tty
database dir   : /var/lib/rabbitmq/mnesia/rabbit@0a0dd6de799a

=INFO REPORT==== 2-Jun-2016::21:05:28 ===
autocluster: Delaying startup for 4223ms.

=INFO REPORT==== 2-Jun-2016::21:05:32 ===
autocluster: Starting consul registration.

=INFO REPORT==== 2-Jun-2016::21:05:32 ===
autocluster: Registering node with consul.

=INFO REPORT==== 2-Jun-2016::21:05:32 ===
Memory limit set to 1579MB of 3949MB total.

=INFO REPORT==== 2-Jun-2016::21:05:32 ===
Disk free limit set to 50MB

=INFO REPORT==== 2-Jun-2016::21:05:32 ===
Limiting to approx 1048476 file handles (943626 sockets)

=INFO REPORT==== 2-Jun-2016::21:05:32 ===
FHC read buffering:  OFF
FHC write buffering: ON

=INFO REPORT==== 2-Jun-2016::21:05:32 ===
Database directory at /var/lib/rabbitmq/mnesia/rabbit@0a0dd6de799a is empty. Initialising from scratch...

=INFO REPORT==== 2-Jun-2016::21:05:32 ===
    application: mnesia
    exited: stopped
    type: temporary

=INFO REPORT==== 2-Jun-2016::21:05:32 ===
Priority queues enabled, real BQ is rabbit_variable_queue

=INFO REPORT==== 2-Jun-2016::21:05:32 ===
Management plugin: using rates mode 'basic'

=INFO REPORT==== 2-Jun-2016::21:05:32 ===
msg_store_transient: using rabbit_msg_store_ets_index to provide index

=INFO REPORT==== 2-Jun-2016::21:05:32 ===
msg_store_persistent: using rabbit_msg_store_ets_index to provide index

=WARNING REPORT==== 2-Jun-2016::21:05:32 ===
msg_store_persistent: rebuilding indices from scratch

=INFO REPORT==== 2-Jun-2016::21:05:32 ===
Adding vhost '/'

=INFO REPORT==== 2-Jun-2016::21:05:32 ===
Creating user 'guest'

=INFO REPORT==== 2-Jun-2016::21:05:32 ===
Setting user tags for user 'guest' to [administrator]

=INFO REPORT==== 2-Jun-2016::21:05:32 ===
Setting permissions for 'guest' in '/' to '.*', '.*', '.*'

=INFO REPORT==== 2-Jun-2016::21:05:32 ===
started TCP Listener on [::]:5672

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Management plugin started. Port: 15672

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics event collector started.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics channel stats collector started.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics queue stats collector started.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics database started.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_queue_stats_fine_stats,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_queue_stats_deliver_get,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_queue_stats_queue_msg_counts,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_queue_stats_queue_msg_rates,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_queue_exchange_stats_fine_stats,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_vhost_stats_deliver_get,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_vhost_stats_fine_stats,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_vhost_stats_queue_msg_rates,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_vhost_stats_queue_msg_counts,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_vhost_stats_coarse_conn_stats,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_channel_queue_stats_deliver_get,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_channel_queue_stats_fine_stats,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_channel_queue_stats_queue_msg_counts,
                                                5000}.
 completed with 8 plugins.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Server startup complete; 8 plugins started.
 * rabbitmq_management
 * rabbitmq_web_dispatch
 * webmachine
 * mochiweb
 * rabbitmq_management_agent
 * amqp_client
 * autocluster
 * rabbitmq_aws

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_channel_stats_deliver_get,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_channel_stats_fine_stats,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_channel_stats_queue_msg_counts,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_channel_exchange_stats_deliver_get,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_channel_exchange_stats_fine_stats,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_exchange_stats_fine_stats,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_node_stats_coarse_node_stats,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_node_node_stats_coarse_node_node_stats,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {aggr_connection_stats_coarse_conn_stats,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {channel_stats,5000}.

=INFO REPORT==== 2-Jun-2016::21:05:33 ===
Statistics garbage collector started for table {connection_stats,5000}.

and

$ docker logs 90eb3ebedfac

=ERROR REPORT==== 2-Jun-2016::21:05:25 ===
Failed to retrieve AWS credentials: undefined

=INFO REPORT==== 2-Jun-2016::21:05:25 ===
Starting RabbitMQ 3.6.2 on Erlang 18.3
Copyright (C) 2007-2016 Pivotal Software, Inc.
Licensed under the MPL.  See http://www.rabbitmq.com/

              RabbitMQ 3.6.2. Copyright (C) 2007-2016 Pivotal Software, Inc.
  ##  ##      Licensed under the MPL.  See http://www.rabbitmq.com/
  ##  ##
  ##########  Logs: tty
  ######  ##        tty
  ##########
              Starting broker...

=INFO REPORT==== 2-Jun-2016::21:05:25 ===
node           : rabbit@90eb3ebedfac
home dir       : /var/lib/rabbitmq
config file(s) : /etc/rabbitmq/rabbitmq.config
cookie hash    : avIdrkSVmb3GWEFtBz8hvA==
log            : tty
sasl log       : tty
database dir   : /var/lib/rabbitmq/mnesia/rabbit@90eb3ebedfac

=INFO REPORT==== 2-Jun-2016::21:05:28 ===
autocluster: Delaying startup for 2649ms.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
autocluster: Starting consul registration.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
autocluster: Registering node with consul.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Memory limit set to 1579MB of 3949MB total.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Disk free limit set to 50MB

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Limiting to approx 1048476 file handles (943626 sockets)

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
FHC read buffering:  OFF
FHC write buffering: ON

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Database directory at /var/lib/rabbitmq/mnesia/rabbit@90eb3ebedfac is empty. Initialising from scratch...

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
    application: mnesia
    exited: stopped
    type: temporary

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Priority queues enabled, real BQ is rabbit_variable_queue

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Management plugin: using rates mode 'basic'

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
msg_store_transient: using rabbit_msg_store_ets_index to provide index

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
msg_store_persistent: using rabbit_msg_store_ets_index to provide index

=WARNING REPORT==== 2-Jun-2016::21:05:31 ===
msg_store_persistent: rebuilding indices from scratch

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Adding vhost '/'

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Creating user 'guest'

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Setting user tags for user 'guest' to [administrator]

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Setting permissions for 'guest' in '/' to '.*', '.*', '.*'

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
started TCP Listener on [::]:5672

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Management plugin started. Port: 15672

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics event collector started.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics channel stats collector started.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics queue stats collector started.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics database started.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_queue_stats_fine_stats,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_queue_stats_deliver_get,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_queue_stats_queue_msg_counts,
                                                5000}.
 completed with 8 plugins.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_queue_stats_queue_msg_rates,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Server startup complete; 8 plugins started.
 * rabbitmq_management
 * rabbitmq_web_dispatch
 * webmachine
 * mochiweb
 * rabbitmq_management_agent
 * amqp_client
 * autocluster
 * rabbitmq_aws

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_queue_exchange_stats_fine_stats,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_vhost_stats_deliver_get,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_vhost_stats_fine_stats,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_vhost_stats_queue_msg_rates,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_vhost_stats_queue_msg_counts,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_vhost_stats_coarse_conn_stats,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_channel_queue_stats_deliver_get,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_channel_queue_stats_fine_stats,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_channel_queue_stats_queue_msg_counts,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_channel_stats_deliver_get,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_channel_stats_fine_stats,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_channel_stats_queue_msg_counts,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_channel_exchange_stats_deliver_get,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_channel_exchange_stats_fine_stats,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_exchange_stats_fine_stats,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_node_stats_coarse_node_stats,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_node_node_stats_coarse_node_node_stats,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {aggr_connection_stats_coarse_conn_stats,
                                                5000}.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {channel_stats,5000}.

=INFO REPORT==== 2-Jun-2016::21:05:31 ===
Statistics garbage collector started for table {connection_stats,5000}.
digitalkaoz commented 8 years ago

both nodes can reach each other:

root@0a0dd6de799a:/# dig rabbitmq.service.consul

; <<>> DiG 9.9.5-9+deb8u6-Debian <<>> rabbitmq.service.consul
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 13360
;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;rabbitmq.service.consul.   IN  A

;; ANSWER SECTION:
rabbitmq.service.consul. 0  IN  CNAME   0a0dd6de799a.
rabbitmq.service.consul. 0  IN  CNAME   90eb3ebedfac.
digitalkaoz commented 8 years ago

but: they are not accessible by nodename, just by ip:

root@0a0dd6de799a:/# telnet 90eb3ebedfac 5672
telnet: could not resolve 90eb3ebedfac/5672: Name or service not known
root@0a0dd6de799a:/# telnet 172.17.0.8 5672
Trying 172.17.0.8...
Connected to 172.17.0.8.
Escape character is '^]'.
^C^CConnection closed by foreign host.
root@0a0dd6de799a:/# telnet 172.17.0.9 5672
Trying 172.17.0.9...
Connected to 172.17.0.9.
Escape character is '^]'

so i guess it could be fixed by setting the ip in consul instead of the hostname?

gmr commented 8 years ago

Cookies match, which is good :)

What about dig 90eb3ebedfac on that container?

Also, adding:

{log_levels, [{autocluster, debug}]}

to your RabbitMQ config could help straighten out what's up. Are you using the aweber/rabbitmq-autocluster image, or your own? I should make a way to turn debugging logs on easier.

gmr commented 8 years ago

Hmm what about 90eb3ebedfac.rabbitmq.service.consul?

digitalkaoz commented 8 years ago

nope that wont work, i guess

CONSUL_SVC_ADDR_AUTO should be able to propagate the ip instead of the hostname, but how to get the right ip?

as im using docker, i dont know which ip the container gets

digitalkaoz commented 8 years ago

CONSUL_SVC_ADDR would be right if i only knew the ip on startup so i can set this env var

digitalkaoz commented 8 years ago

hostname --ip-address would be right i think!

digitalkaoz commented 8 years ago

maybe you can add an environment variable which decides if CONSUL_SVC_ADDR_AUTO exposes the hostname or the ip, maybe CONSUL_SVC_ADDR_IP=true|false

digitalkaoz commented 8 years ago

But that might be problematic in overlay networks with multiple public ips. Best would be elasticsearch s solution: https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html#network-interface-values define an network interface to get the ip from....

Sadly im not familiar with erlang, otherwise i would help :)

Gavin M. Roy notifications@github.com schrieb am Do., 2. Juni 2016, 23:27:

Hmm what about 90eb3ebedfac.rabbitmq.service.consul?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/aweber/rabbitmq-autocluster/issues/60#issuecomment-223428047, or mute the thread https://github.com/notifications/unsubscribe/AAR615e7sL5c1h68GerbLo7_Zl01kV1kks5qH0rVgaJpZM4IsXpv .

gmr commented 8 years ago

Hmm ok, that sounds reasonable -- I was debating over that when I wrote the code for the ADDR_AUTO; unfortunately I scrapped my code that parsed the data structure that returns it. The first version of ADDR_AUTO used IP address.

digitalkaoz commented 8 years ago

Maybe add it back and let the User decide? :)

Gavin M. Roy notifications@github.com schrieb am Fr., 3. Juni 2016, 00:27:

Hmm ok, that sounds reasonable -- I was debating over that when I wrote the code for the ADDR_AUTO; unfortunately I scrapped my code that parsed the data structure that returns it. The first version of ADDR_AUTO used IP address.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/aweber/rabbitmq-autocluster/issues/60#issuecomment-223441313, or mute the thread https://github.com/notifications/unsubscribe/AAR617mfEzp6ewetoo6iw8IWb1rsCdFiks5qH1jHgaJpZM4IsXpv .

gmr commented 8 years ago

I've added CONSUL_SVC_ADDR_NIC for 0.6.0 and am gearing up to release that now.

gmr commented 8 years ago

0.6.0 has been released. I'll close this issue out, feel free to re-open if you continue to have an issue.

digitalkaoz commented 8 years ago

You are awesome Dude! Will test it later today and give you some Feedback :)

Gavin M. Roy notifications@github.com schrieb am Sa., 4. Juni 2016, 01:07:

Closed #60 https://github.com/aweber/rabbitmq-autocluster/issues/60.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/aweber/rabbitmq-autocluster/issues/60#event-681755853, or mute the thread https://github.com/notifications/unsubscribe/AAR61yHbeR2_HE6lwLEngGBpwXPwcKxFks5qILPMgaJpZM4IsXpv .