att-comdev / openstack-helm

PROJECT HAS MOVED TO OPENSTACK
https://github.com/openstack/openstack-helm
69 stars 41 forks source link

bug: Charts using invalid rabbitmq credentials by default #239

Closed mattmceuen closed 7 years ago

mattmceuen commented 7 years ago

Is this a bug report or feature request? (choose one): Bug report

Kubernetes Version (output of kubectl version): Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"08e099554f3c31f6e6f07b448ab3ed78d0520507", GitTreeState:"clean", BuildDate:"2017-01-12T04:57:25Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.1", GitCommit:"82450d03cb057bab0950214ef122b67c83fb11df", GitTreeState:"clean", BuildDate:"2016-12-14T00:52:01Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}

Helm Client and Tiller Versions (output of helm version): Client: &version.Version{SemVer:"v2.1.3", GitCommit:"5cbc48fb305ca4bf68c26eb8d2a7eb363227e973", GitTreeState:"clean"} Server: &version.Version{SemVer:"v2.1.3", GitCommit:"5cbc48fb305ca4bf68c26eb8d2a7eb363227e973", GitTreeState:"clean"}

Development or Deployment Environment?: Single node bare metal dev

Release Tag or Master: Master

Expected Behavior: OpenStack services can communicate with rabbitmq

What Actually Happened: A number of services fail to connect to rabbitmq, with the error: AccessRefused: (0, 0): (403) ACCESS_REFUSED - Login was refused using authentication mechanism AMQPLAIN. For details see the broker logfile.

How to Reproduce the Issue (as minimally as possible): Do a fresh install from master, per the install guide.

Any Additional Comments: Several pods get the error above: neutron-dhcp-agent neutron-l3-agent cinder-scheduler nova-compute nova-consoleauth nova-scheduler

These services are configured by default like so: rabbitmq: address: rabbitmq admin_user: rabbitmq admin_password: password

Rabbitmq reports the corresponding error: AMQPLAIN login refused: user 'rabbitmq' - invalid credentials

Rabbitmq's values has: auth: default_user: openstack default_pass: password admin_user: rabbitmq admin_pass: password

However, the admin_user/admin_pass are not actually included in a template. So, rabbitmq is not configured by default to use the "rabbitmq" user that the OpenStack services are trying to use.

mattmceuen commented 7 years ago

There are some different ways to resolve this, and I'm not sure which is best: 1) Change rabbitmq's default_user to "rabbitmq", like the openstack components expect.
2) Change the openstack components' rabbit login to "openstack", like rabbitmq expects. 3) Somehow template the admin_user in the rabbitmq values.yaml to actually configure something appropriate -- it's not clear to me what that is; there doesn't appear to be an "admin_user" setting in the rabbitmq config 1.

intlabs commented 7 years ago

@mattmceuen this is indeed rather ugly at present, I'd like to see us move in the direction of using proper users for rabbitmq. Until then, unless anyone objects I think option 1 makes the most sense.

jallirs commented 7 years ago

I would say the best is to have randomly generated values for the password/username inside values.yaml by default that can be overridden.

And then for the interim case I agree with Pete. Option 1. On Sat, Feb 25, 2017 at 11:18 PM Pete Birley notifications@github.com wrote:

@mattmceuen https://github.com/mattmceuen this is indeed rather ugly at present, I'd like to see us move in the direction of using proper users for rabbitmq. Until then, unless anyone objects I think option 1 makes the most sense.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/att-comdev/openstack-helm/issues/239#issuecomment-282531872, or mute the thread https://github.com/notifications/unsubscribe-auth/ABfutGRpTe7zdCby2ENdFiqoHRs4nLq2ks5rgP0MgaJpZM4MMLI8 .

v1k0d3n commented 7 years ago

@jallirs stronger passwords can be worked out later, when we can also reliably present the password artifact to users. there are likely a couple of ways to do this. but i'm with @intlabs, starting with sane users is a good first step and a quick, iterative approach.

jallirs commented 7 years ago

Yeah I agree. :) I think for the interim case we're on the same page :) On Mon, Feb 27, 2017 at 8:33 AM Brandon B. Jozsa notifications@github.com wrote:

@jallirs https://github.com/jallirs stronger passwords can be worked out later, when we can also reliably present the password artifact to users. there are likely a couple of ways to do this. but i'm with @intlabs https://github.com/intlabs, starting with sane users is a good first step and a quick, iterative approach.

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/att-comdev/openstack-helm/issues/239#issuecomment-282720677, or mute the thread https://github.com/notifications/unsubscribe-auth/ABfutIoL_85YChBsdw96IgP6xyke8jZKks5rgtC4gaJpZM4MMLI8 .

intlabs commented 7 years ago

@jallirs Using password everywhere is indeed about as far from ideal as you could get. Something we will need to address moving forward is how to set secure passwords for services by default. Personally, I'm leaning toward proper documentation that makes it clear that users can deploy either with a customized values.yaml or using commands like:

RABBIT_PASS=$(openssl rand -base64 32)
helm install local/rabbitmq \
    --name=rabbitmq \
    --namespace=openstack \
    --set auth.default_user=rabbitmq,auth.default_pass=${RABBIT_PASS}

But I'm very keen to get any other suggestions, and we should definitely start to think about this as a group before too long.

v1k0d3n commented 7 years ago

i think this is the best way to address this, honestly. otherwise, if we try to do it in code things will quickly get messy (something by nature and vision, we're trying to avoid). i'm not saying that we're going to be able to fight this off forever, but we should try...and clearly documenting this operations case will give us something to point when operators bring it up. just my 0.2.

alanmeadows commented 7 years ago

Another value add of a wrapper like armada (or even a helm plugin potentially), and nest the intent right in the YAML:

values:
  auth:
    default_pass: !!python/object/apply:armada.utils.gen_password(length=8)