clustering: erlang cookie ownership

[KeithWoods]

I'm trying to get clustering working and noticed I was getting this error on the node that first ran the cluster operation:

rabbit2_1  | start_error
rabbit2_1  | {"Error when reading /var/lib/rabbitmq/.erlang.cookie: eacces",[{auth,init_cookie,0,[{file,"auth.erl"},{line,286}]},{auth,init,1,[{file,"auth.erl"},{line,140}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,328}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,247}]}]}
rabbit2_1  | [{pid,undefined},{id,auth},{mfargs,{auth,start_link,[]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]
rabbit2_1  | {local,kernel_sup}

my script is doing this:

/usr/sbin/rabbitmq-server -detached
rabbitmqctl stop_app
rabbitmqctl join_cluster rabbit@$someNode

If I add in the below before I interact with rabbit it seems to work fine.

chown rabbitmq:rabbitmq /var/lib/rabbitmq/.erlang.cookie

Should the above chown be under this line? https://github.com/docker-library/rabbitmq/blob/29121864d4892b2481706df023a53e31fececd02/docker-entrypoint.sh#L139

[yosifkit]

It shouldn't be needed (and indeed would fail), since we are running as the rabbitmq user (or another non-root user if --user was used on the docker run) after the gosu on line 9.

[KeithWoods]

Thanks for the reply. Given my behaviour isn't expected let me try recreate the steps and report back else this can be closed.

[seirdb]

I've had some occasional crashes with my rabbitmq container with the same error message. If I run the following command, it sometimes raises this issue (after about 50 tries or so)

docker run -d --hostname rabbit --name rabbitmq-host -p 8080:15672 -p 5672:5672 rabbitmq:3.6.12-management && docker exec -i rabbitmq-host rabbitmqctl wait /var/lib/rabbitmq/mnesia/rabbit@rabbit.pid

If I reboot the host the containers are running on, it always results in a crashing container for the first time the command is executed.

My guess is that the daemonized container returns and then quickly the host tries to docker exec the rabbitmqctl command, which runs against the container daemon not having been able to update the permissions on the cookie?

[seirdb]

Btw, adding --user rabbitmq to both run and exec seems to work as expected and doesn't crash the container.

[tianon]

Closing given that this appears to be resolved and/or environmental. If this can be reproduced, a new issue outlining the exact steps we can take to reproduce would be grand. Thanks!

[kasir-barati]

To be perfectly honest I am not sure if it is really reproducible since I've encountered it and it is just randomly poping up. But I think this might be useful for those who are using docker-compose and env variables: https://github.com/kasir-barati/nestjs-materials/blob/2cbb5f5fdea0876d7a809bd75bd255726e4c918b/microservices/complex-reservation-rabbitmq/compose.yaml

In a nutshell I've added:

1. In my compose.yaml:

   services:
     # ...
     message-brocker:
       image: rabbitmq:4.0-rc-management-alpine
       # ...
       user: ${RABBITMQ_DEFAULT_USER}

2. Then in my .env file:

   RABBITMQ_URI=amqp://rabbitmq:password@message-brocker:5672
   RABBITMQ_DEFAULT_USER=rabbitmq
   RABBITMQ_DEFAULT_PASS=password

Update 2024.09.10

I bumped into this again today. And even though I just copied and pasted my previous docker compose file it did not work. So first thing I realized was that I was using "root" as a value for RABBITMQ_DEFAULT_USER. So I changed it to "rabbitmq" and removed the containers altogether (docker compose down -v). And now everything was working just fine.