search

rabbitmq / rabbitmq-autocluster

RabbitMQ peer discovery and cluster formation plugin, supports RabbitMQ 3.6.x
BSD 3-Clause "New" or "Revised" License
242 stars 54 forks source link

Retry in case create_session gets error 500 #44

Closed Gsantomaggio closed 7 years ago

Gsantomaggio commented 7 years ago

Fixes https://github.com/rabbitmq/rabbitmq-autocluster/issues/42

When Consul is starting, the sockets are up, but the end points are not ready yet

Consul raises the following error when the plugin tries to create a session:

2017/08/22 08:55:02 [ERR] http: Request PUT /v1/session/create, error: No cluster leader from=127.0.0.1:58446

And the broker gets down:

=INFO REPORT==== 22-Aug-2017::08:47:36 ===
Error description:
   {could_not_start,rabbit,
       {{case_clause,"Error while creating a session, reason: \"500\""},
        [{autocluster,acquire_startup_lock,1,
             [{file,"src/autocluster.erl"},{line,212}]},
         {autocluster,run_steps,1,[{file,"src/autocluster.erl"},{line,131}]},

With this fix, in case the session creation raises Error 500 it retries to execute the create session.

To reproduce the issue:

  1. Run a clean RabbitMQ instance
  2. When is "Starting broker.." start a clean Consul instance