Add support for custom TLS based URL in Topology

[ShadiAlbatal]

We are deploying a rabbitmq cluster with TLS, and we an ingress role to route it over HTTPS. The cluster creates by default a secret named <rabbit-name>-default-user, which contains user, pass, host,..etc.

First, the host value <rabbit-name>.<ns-name>.svc and not my custom host which is ex: rabbits.example.com

So, when i deploy the Topology, and try to create an Exchange object, ex:

apiVersion: rabbitmq.com/v1beta1
kind: Exchange
metadata:
  name: fanout
  namespace: ns-name
spec:
  name: fanout-exchange
  type: fanout
  rabbitmqClusterReference:
    name: rabbitmq-name
    namespace: rabbitmq-ns

it tried to connect using the host in the secret, which is wrong and it fails (see attache image).

Now if i handle the situation and create a secret then reference it in the object, such as:

apiVersion: v1
kind: Secret
metadata:
  name: my-rabbit-creds
type: Opaque
stringData:
  username: a-user
  password: a-secure-password
  uri: https://rabbits.example.com
---
apiVersion: rabbitmq.com/v1beta1
kind: Queue
metadata:
  name: qq-example
spec:
  name: my-queue
  rabbitmqClusterReference:
    connectionSecret:
      name: my-rabbit-creds

This will work, but then i cannot use the predefined user/pass which is generated by the cluster (i can if i hardcode it but i dont want to).

That is it.

So i think, you:

1. either need to make the secret to pick up the custom host
2. or to make a way to extract the user/pass, so when i create my own secret, i can reference user/pass dynamically.

Note: One more thing for you to write in the docs or check/fix: is that when using 15671, we cannot write it in the secret as: uri: https://rabbits.example.com:15671 it does not work, i must delete the port and write uri: https://my.rabbit to get it working.

I tried to make as clear as possible, hope it is clearly explained.

[Zerpet]

This remind me of https://github.com/rabbitmq/messaging-topology-operator/issues/233. Is the domain in the certificate (e.g. .example.com) also the internal Kubernetes domain? In other words, would this work for you?

We could make the Topology Operator consume a pre-determined annotation, where the value of this annotation would be an external URL to connect to a particular RabbitmqCluster e.g. rabbitmq.example.com.

[ShadiAlbatal]

Answering your question: the certificate is issued for the domain, but i dont think it is made for the internal domain. the tls is made the the domain, but the cluster internal domain is still as is

I guess the issue you refered too is something i can look at for now, i just understood how it works, however it would be nice if we skip patching topology operator for the domain or the ca.crt because i am using terrafrom and an operator manager to deploy operator and topology and i am trying to avoid patching

[github-actions[bot]]

This issue has been marked as stale due to 60 days of inactivity. Stale issues will be closed after a further 30 days of inactivity; please remove the stale label in order to prevent this occurring.

[Vedrillan]

Among all the similar issues, I found no proper solution to this problem. Issue is it seems it is impossible to have a cluster publicly exposed with TLS enabled and fully managed by the topology operator.

Changing MESSAGING_DOMAIN_NAME assumes we want to expose all our clusters to the public network, which is not always the case, we need to be able to have cluster only as an internal k8s service as well. And manual secrets removes the benefit of having managed clusters.

I think the right solution would be to add the ability to customize the topology endpoint of a managed cluster, so that instead of using the generated private k8s service (e.g. {cluster-name}.{namespace}.svc) we can use a public endpoint (e.g. my-rabbitmq-cluster.example.com)

[ChunyiLyu]

@Vedrillan Briefly looked at the issue and just opened a PR in topology operator that allows custom URI to be set by annotating rabbitmqcluster: https://github.com/rabbitmq/messaging-topology-operator/pull/644

Annotation key is set to operator-connection-uri at the moment. This way you won't need to create an additional secret object just to configure the connection URI. Does this work for your use case? You can test this change via this docker image rabbitmqoperator/messaging-topology-operator-dev:chunyi-test.

[Vedrillan]

Like so right ?

apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:
  name: rabbitmqcluster-sample
  annotations:
    operator-connection-uri: rabbitmqcluster-sample.example.com

[ChunyiLyu]

@Vedrillan yes that's the idea. An annotation on rabbitmqcluster to tell topology operator to use the specified URI instead of the k8s dns. No need to create a secret object, because the operator will still use the default user secret credentials to connect. Any thoughts?

[Zerpet]

I think this is great! I would suggest to add a suffix to the annotation, to avoid collisions, and to explicitly "mark" the annotation as rabbitmq specific. A suffix like rabbitmq.com/ should do it. Similar to what AWS Ingress controller does e.g. alb.ingress.kubernetes.io/group.name: my-group

[ChunyiLyu]

https://github.com/rabbitmq/messaging-topology-operator/pull/644 is merged and the annotation key is rabbitmq.com/operator-connection-uri. The feature will be released in the coming topology operator release.