I upgraded a cluster from 3.7.11 to 3.8.5 today. Upon starting 3.8.5, and after some investigation, it appears to be ignoring the /etc/rabbitmq/rabbitmq-env.conf file, which has RABBITMQ_MNESIA_BASE set to the location of the mnesia database. As a result it's trying to find the database in /var/lib/rabbitmq, and then creating a new database there.
Has something changed? I've checked through the website and from what I'm reading the location of the rabbitmq-env.conf file hasn't changed, nor has this variable name. For now I've had to roll back go 3.7.11 to get things working again.
From @dumbbell -
My analysis of the problem: before soucing the env file in rabbit_env, we pass it all variables it can expect (setting them to default values). $MNESIA_BASE is among them. When we read the result of the script, we handle variable in alphabetical order. The script returns both $MNESIA_BASE and $RABBITMQ_MNESIA_BASE in the case of those end-users: the former because we set it before calling it, the latter because the file defines it. Because of the alphabetical order, $MNESIA_BASE is handled first: it is not in the environment, so we export it (so rabbit_env picks it). When we handle $RABBITMQ_MNESIA_BASE, we check if the variable is already exported (which is the case because of $MNESIA_BASE), thus it is ignored.
https://vmware.slack.com/archives/C055BSG8E/p1594049782287100
https://rabbitmq.slack.com/archives/C1EDN83PA/p1594046979401900
From @dumbbell -