mongodb / mongodb-kubernetes-operator

MongoDB Community Kubernetes Operator
Other
1.22k stars 500 forks source link

How to expose mongodb externally outside of kubernetes cluster #127

Closed kaykhancheckpoint closed 3 years ago

kaykhancheckpoint commented 4 years ago

Hi i want to expose the mongodb publicly. But im not sure where to set mongodb service as type load balancer?

Can anyone provide an example of this, thanks

navndn commented 4 years ago

Hey @kaykhancheckpoint did you figure out?

I'm thinking of adding an Istio ServiceEntry for mongodb's clusterIP service and then exposing the ServiceEntry through Istio's Gateway+VirtualService... Not sure this will workout or even if it does, there must be an elegant way than this...

kaykhancheckpoint commented 4 years ago

@navndn No i didn't. I've moved away from this operator for the time being, its still in pre-release and there are some important features that it doesn't have right now which i need.

For now im using mongodb helm chart owned by bitnami.

But, I'll keep an eye on this and see how it progresses, maybe ill try it again in the future once there is a stable release

navndn commented 4 years ago

@kaykhancheckpoint thanks for response. Even I decided to use bitnami chart for now

LouisPlisso commented 4 years ago

@kaykhancheckpoint thanks for your question

External access

You can configure your mongodb deployment to be reachable from outside the kubernetes cluster with a LoadBalancer or NodePort (like with the bitnami helm chart). We don't provide examples as of now, as we do believe all connections should be over SSL and it's not straightforward to provide SSL encrypted connections from outside the cluster.

We do have a solution for external access to the cluster with SSL currently implemented with the enterprise operator and we will provide it soon to the community/open-source operator (this repo): it is based on split horizon, see point 8 here for how to configure in enterprise https://docs.mongodb.com/kubernetes-operator/master/tutorial/deploy-replica-set/.

Important features

@kaykhancheckpoint @navndn could you please give us a list of important features you need so that we can incorporate them to our roadmap? Either here or on https://feedback.mongodb.com/forums/924355-ops-tools.

FYI we plan to have all the authentication and encryption features are implemented (including split-horizon) within a month, you can follow the progress here: https://github.com/mongodb/mongodb-kubernetes-operator/projects/2.

guraybaydur commented 4 years ago

Hi everyone, I have a question related to this issue.

I deployed mongodb community operator on Openshift. I can connect to my resources from outside via NodePort and LoadBalancer option but I can not connect it via routes (either secure or insecure).

when i try to connect it gives this error: "Error: network error while attempting to run command 'isMaster' on host '"

Is there a way to use route without ssl even though I know using ssl is the preferred way?

Thank you!

guraybaydur commented 4 years ago

Hi @LouisPlisso , hope you are good.

I deployed mongodb community operator on Openshift. I can connect to my resources from outside via NodePort and LoadBalancer. But the problem is sometimes these service resources direct me to secondary mongodb resource instead of primary and I can not run any query on secondary resource because it gives the error below:

Screen Shot 2020-10-08 at 10 13 45

Is there a way to always direct connections to primary resource?

Thank you

guraybaydur commented 4 years ago

Hi again @chatton , I would like to ask you my last question.

I can access my mongodb replica sets from outside cluster via NodePort or LoadBalancer but they sometimes direct connection to primary and sometimes to secondary. I would like to ask if there is a way to know which pod is exactly primary (I deleted the pods and they switched roles in some occassions) so that I won't face any NotMasterNoSlaveOk problem.

I learnt that I can send "mongo ... --eval rs.slaveOk=true ..." while connecting to use secondary without any denial but is there a way to set slaveOk=true from any config file. I searched internet people talked about editing "/etc/mongorc.js" file but our pods don't have that directory.

Could you please help me with this issue? I am stuck in this phase.

Thank you very much!

yjwong commented 4 years ago

I was able to get this operator to work with some modifications, using the (undocumented) split horizon functionality available in MongoDB. This uses SNI, thus does require TLS to be enabled; if you have already have a PKI of sorts it's a lot easier.

In summary, besides the default Service that the operator creates, I created 3 more Service objects (type NodePort) referencing the statefulset.kubernetes.io/pod-name label, then created DNS records referencing their external IP addresses.

Then, configure the MongoDB resource like so:

apiVersion: mongodb.com/v1
kind: MongoDB
metadata:
  name: production
  namespace: mongodb-operator
spec:
  members: 3
  replicaSetHorizons:
  - horizon: production-0.mongodb-operator.example.com:31181
  - horizon: production-1.mongodb-operator.example.com:31182
  - horizon: production-2.mongodb-operator.example.com:31183
  # ...

(the replicaSetHorizons was added by my patch - it won't work with the current operator!)

guraybaydur commented 4 years ago

Hi @yjwong ,

I am trying your fork for accessing externally. But I am having an error in stateful set where image name is set incorrectly.

Screen Shot 2020-10-20 at 23 36 22

Screen Shot 2020-10-20 at 23 42 13

Screen Shot 2020-10-20 at 23 41 49

Do you know how can we fix this problem? (btw 4.2.6 or 4.2.7 either definition in "...cr.yaml" not works)

inscapist commented 3 years ago

This uses SNI, thus does require TLS to be enabled

Thanks @yjwong for the pointer. I almost spent a day for this.

Just sharing the steps below to help others and most likely my future self.

1) Import CA cert and server key(pem)

Generate the certificates with something like mkcert. Refer this

Import them as configmap and secret, respectively.

2) Create NodePort services

30000,30001,30002 will be available on all nodes.

---
apiVersion: v1
kind: Service
metadata:
  name: example-mongodb-instance-0
spec:
  type: NodePort
  ports:
    - port: 27017
      targetPort: 27017
      nodePort: 30000
  selector:
    statefulset.kubernetes.io/pod-name: example-mongodb-0
---
apiVersion: v1
kind: Service
metadata:
  name: example-mongodb-instance-1
spec:
  type: NodePort
  ports:
    - port: 27017
      targetPort: 27017
      nodePort: 30001
  selector:
    statefulset.kubernetes.io/pod-name: example-mongodb-1
---
apiVersion: v1
kind: Service
metadata:
  name: example-mongodb-instance-2
spec:
  type: NodePort
  ports:
    - port: 27017
      targetPort: 27017
      nodePort: 30002
  selector:
    statefulset.kubernetes.io/pod-name: example-mongodb-2

3) Create SRV and TXT records

rs.mongo.k3s is what we will use in connection string later Screenshot is taken from a private hosted zone

records

4) Specify split horizon DNS and TLS

Include the spec replicaSetHorizons. The key "horizon" here does not matter. Also ensure certificateKeySecretRef and caConfigMapRef is using the ones that you have created.

spec:
  members: 3
  replicaSetHorizons:
    - horizon: "nodes.mongo.k3s:30000"
    - horizon: "nodes.mongo.k3s:30001"
    - horizon: "nodes.mongo.k3s:30002"
  type: ReplicaSet
  version: "4.2.6"
  security:
    tls:
      enabled: true
      certificateKeySecretRef:
        name: mongo-tls-cert # create
      caConfigMapRef:
        name: mongo-ca
    authentication:
      modes: ["SCRAM"]

5) Connect

mongo mongodb+srv://USER:PASS@rs.mongo.k3s/
vasRK commented 3 years ago

` apiVersion: v1 kind: Service metadata: annotations: service.beta.kubernetes.io/azure-dns-label-name: example-mongodb-prod name: example-mongodb-prod-lb spec: loadBalancerIP: 111.111.111.111 type: LoadBalancer selector: statefulset.kubernetes.io/pod-name: example-mongodb-0 ports:

`

I did it with a cluster loadbalancer, i use azure kubernetes service, im able to access mongodb using assigned dns name or ip both from mongo shell & mongo compass

dgdevops commented 3 years ago

Hello @yjwong & @sagittaros Thank you for devoting the time to adding & testing the ReplicasetHorizons feature. I have upgraded to the latest version of the operator, however for some reason I still cannot reach the ReplicaSet from external sources:

/ # mongo "mongodb://public-ip-0:27017,public-ip-1:27017,public-ip-2:27017/?replicaSet=rs-name" --ssl --sslAllowInvalidCertificates
MongoDB shell version v4.0.5
connecting to: mongodb://public-ip-0:27017,public-ip-1:27017,public-ip-2:27017/?gssapiServiceName=mongodb&replicaSet=rs-name
2021-07-06T12:17:47.715+0000 I NETWORK  [js] Starting new replica set monitor for rs-name/public-ip-0:27017,public-ip-1:27017,public-ip-2:27017
2021-07-06T12:17:47.729+0000 W NETWORK  [ReplicaSetMonitor-TaskExecutor] SSL peer certificate validation failed: self signed certificate
2021-07-06T12:17:47.730+0000 W NETWORK  [js] SSL peer certificate validation failed: self signed certificate
2021-07-06T12:17:47.733+0000 I NETWORK  [ReplicaSetMonitor-TaskExecutor] Successfully connected to public-ip-0:27017 (1 connections now open to public-ip-0:27017 with a 5 second timeout)
2021-07-06T12:17:47.734+0000 I NETWORK  [js] Successfully connected to public-ip-1:27017 (1 connections now open to public-ip-1:27017 with a 5 second timeout)
2021-07-06T12:17:47.812+0000 I NETWORK  [js] changing hosts to rs-name/rs-name-0.rs-name-svc.mongodb.svc.cluster.local:27017,rs-name/rs-name-1.rs-name-svc.mongodb.svc.cluster.local:27017,rs-name/rs-name-2.rs-name-svc.mongodb.svc.cluster.local:27017:27017 from rs-name/public-ip-2:27017,public-ip-1:27017,public-ip-0:27017
2021-07-06T12:17:48.518+0000 W NETWORK  [js] Unable to reach primary for set rs-name

I have the following configuration in place:

  replicaSetHorizons:
  - horizon: public-ip-0:27017
  - horizon: public-ip-1:27017
  - horizon: public-ip-2:27017

Note: Executing the mongo command with the same connection string on a pod residing on the cluster where the ReplicaSet is deployed results in successful connection. I created 3 services with LoadBalancer type to expose each ReplicaSet members (PRIM-SEC-SEC).

Is there any way to get rid of this error?

inscapist commented 3 years ago

@dgdevops with a quick glance (and with a fuzzy memory), I think the issue lies in the usage of IP addresses where it should have been a list of domains.

External cluster access leverages split horizon, which is a DNS feature.

dgdevops commented 3 years ago

Hello @sagittaros, Thank you for getting back to me so quickly. I forgot to mention that my first attempt was to use domain names in the replicaSetHorizons settings but I stumbled upon the following error:

connecting to: mongodb://cg0.public-domain:27017,cg1.public-domain:27017,cg2.public-domain:27017/?compressors=disabled&gssapiServiceName=mongodb&replicaSet=rs-name
2021-07-07T17:32:05.122+0200 I  NETWORK  [js] Starting new replica set monitor for rs-name/cg0.public-domain:27017,cg1.public-domain:27017,cg2.public-domain:27017
2021-07-07T17:32:05.123+0200 I  CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to cg2.public-domain:27017
2021-07-07T17:32:05.123+0200 I  CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to cg1.public-domain:27017
2021-07-07T17:32:05.124+0200 I  CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to cg0.public-domain:27017
2021-07-07T17:32:05.262+0200 W  NETWORK  [ReplicaSetMonitor-TaskExecutor] SSL peer certificate validation failed: unable to get local issuer certificate
2021-07-07T17:32:05.318+0200 W  NETWORK  [ReplicaSetMonitor-TaskExecutor] SSL peer certificate validation failed: unable to get local issuer certificate
2021-07-07T17:32:05.347+0200 W  NETWORK  [ReplicaSetMonitor-TaskExecutor] SSL peer certificate validation failed: unable to get local issuer certificate
2021-07-07T17:32:05.350+0200 I  CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to rs-name-1.rs-name-svc.mongodb.svc.cluster.local:27017
2021-07-07T17:32:05.400+0200 I  NETWORK  [ReplicaSetMonitor-TaskExecutor] Confirmed replica set for rs-name is rs-name/rs-name-0.rs-name-svc.mongodb.svc.cluster.local:27017,rs-name-1.rs-name-svc.mongodb.svc.cluster.local:27017,rs-name-2.rs-name-svc.mongodb.svc.cluster.local:27017
2021-07-07T17:32:05.407+0200 I  CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to rs-name-0.rs-name-svc.mongodb.svc.cluster.local:27017
2021-07-07T17:32:05.410+0200 I  CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to rs-name-2.rs-name-svc.mongodb.svc.cluster.local:27017
2021-07-07T17:32:05.735+0200 W  NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set rs-name
2021-07-07T17:32:05.736+0200 I  NETWORK  [ReplicaSetMonitor-TaskExecutor] Cannot reach any nodes for set rs-name. Please check network connectivity and the status of the set. This has happened for 1 checks in a row.

Configuration used:

  replicaSetHorizons:
  - horizon: cg0.public-domain:27017
  - horizon: cg1.public-domain:27017
  - horizon: cg2.public-domain:27017

Connectivity test:

# mongo --host cg0.public-domain
MongoDB shell version v4.2.6
connecting to: mongodb://cg0.public-domain:27017/?compressors=disabled&gssapiServiceName=mongodb
MongoDB server version: 4.2.6
rs-name:SECONDARY>

# mongo --host cg1.public-domain
MongoDB shell version v4.2.6
connecting to: mongodb://cg1.public-domain:27017/?compressors=disabled&gssapiServiceName=mongodb
MongoDB server version: 4.2.6
rs-name:PRIMARY>

# mongo --host cg2.public-domain
MongoDB shell version v4.2.6
connecting to: mongodb://cg2.public-domain:27017/?compressors=disabled&gssapiServiceName=mongodb
MongoDB server version: 4.2.6
rs-name:SECONDARY>
inscapist commented 3 years ago

@dgdevops I see some SSL error there, and I suspect you don't have both the internal domains (kubernetes svc) and public domains in your self-signed subjectAltName field. Refer my certificate-ext.conf below.

Ok, let me just share how I did this. Mostly strung together from memory and zsh history so it may not work 100%.

# pre-generated with mkcert
CA_CERT=ca.crt 
CA_KEY=ca.pem

# To be generated
SERVER_PRIVATE_KEY=mongo-key.pem
SERVER_CERT=mongo-crt.pem

# generate server private key (RSA)
openssl genrsa -out ${SERVER_PRIVATE_KEY} 4096

# generate CSR and submitting certificate subject details
openssl req -new \
    -config certificate.conf \
    -key ${SERVER_PRIVATE_KEY} \
    -out csr.pem

# SELF SIGNING HERE
openssl x509 -req \
    -in csr.pem \
    -CA ${CA_CERT} \
    -CAkey ${CA_KEY} \
    -CAcreateserial \
    -out ${SERVER_CERT} \
    -days 3650 \
    -extfile certificate-ext.conf

echo "Your certificate is generated: ${SERVER_CERT}" 

export KUBECONFIG=$HOME/.kube/config
kubectl create configmap mongo-ca --from-file=${CA_CERT}
kubectl create secret tls mongo-tls-cert --cert=${SERVER_CERT} --key=${SERVER_PRIVATE_KEY}

certificate-ext.conf

authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
subjectAltName = @alt_names

[alt_names]
DNS.1 = *.MYAPP-mongodb-svc.default.svc.cluster.local
DNS.2 = nodes.PUBLICDOMAIN.k3s
DNS.3 = rs.PUBLICDOMAIN.k3s
dgdevops commented 3 years ago

Hello @sagittaros , Thank you for getting back to me with an update. I added the public domains to the self-signed certificate as you suggested, the SSL related errors disappeared but the connection is still not possible as seen below:

# mongo "mongodb://cg0.public-domain:27017,cg1.public-domain:27017,cg2.public-domain:27017/?replicaSet=rs-name"
MongoDB shell version v4.2.6
connecting to: mongodb://cg0.public-domain:27017,cg1.public-domain:27017,cg2.public-domain:27017/?compressors=disabled&gssapiServiceName=mongodb&replicaSet=rs-name
2021-07-08T11:44:00.612+0200 I  NETWORK  [js] Starting new replica set monitor for rs-name/cg0.public-domain:27017,cg1.public-domain:27017,cg2.public-domain:27017
2021-07-08T11:44:00.613+0200 I  CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to cg1.public-domain:27017
2021-07-08T11:44:00.613+0200 I  CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to cg0.public-domain:27017
2021-07-08T11:44:00.614+0200 I  CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to cg2.public-domain:27017
2021-07-08T11:44:00.800+0200 I  CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to rs-name-0.rs-name-svc.mongodb.svc.cluster.local:27017
2021-07-08T11:44:00.857+0200 I  NETWORK  [ReplicaSetMonitor-TaskExecutor] Confirmed replica set for rs-name is rs-name/rs-name-0.rs-name-svc.mongodb.svc.cluster.local:27017,rs-name-1.rs-name-svc.mongodb.svc.cluster.local:27017,rs-name-2.rs-name-svc.mongodb.svc.cluster.local:27017
2021-07-08T11:44:00.860+0200 I  CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to rs-name-2.rs-name-svc.mongodb.svc.cluster.local:27017
2021-07-08T11:44:00.866+0200 I  CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to rs-name-1.rs-name-svc.mongodb.svc.cluster.local:27017
2021-07-08T11:44:01.287+0200 W  NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set rs-name
2021-07-08T11:44:01.288+0200 I  NETWORK  [ReplicaSetMonitor-TaskExecutor] Cannot reach any nodes for set rs-name. Please check network connectivity and the status of the set. This has happened for 1 checks in a row.
2021-07-08T11:44:01.763+0200 W  NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set rs-name
2021-07-08T11:44:01.764+0200 I  NETWORK  [ReplicaSetMonitor-TaskExecutor] Cannot reach any nodes for set rs-name. Please check network connectivity and the status of the set. This has happened for 2 checks in a row.
2021-07-08T11:44:02.114+0200 I  CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to rs-name-0.rs-name-svc.mongodb.svc.cluster.local:27017
2021-07-08T11:44:02.115+0200 I  CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to rs-name-1.rs-name-svc.mongodb.svc.cluster.local:27017
2021-07-08T11:44:02.117+0200 I  CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to rs-name-2.rs-name-svc.mongodb.svc.cluster.local:27017
2021-07-08T11:44:02.279+0200 W  NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set rs-name
2021-07-08T11:44:02.279+0200 I  NETWORK  [ReplicaSetMonitor-TaskExecutor] Cannot reach any nodes for set rs-name. Please check network connectivity and the status of the set. This has happened for 3 checks in a row.
2021-07-08T11:44:02.750+0200 W  NETWORK  [ReplicaSetMonitor-TaskExecutor] Unable to reach primary for set rs-name
2021-07-08T11:44:02.750+0200 I  NETWORK  [ReplicaSetMonitor-TaskExecutor] Cannot reach any nodes for set rs-name. Please check network connectivity and the status of the set. This has happened for 4 checks in a row.
^C2021-07-08T11:44:03.134+0200 I  CONTROL  [main] shutting down with code:0

Additionally I also tried connecting to the replicaset from Mongo Compass as well but got the following error: getaddrinfo ENOTFOUND rs-name-0.rs-name-svc.mongodb.svc.cluster.local

The error above makes me think that for some reason the inter-cluster domain names are not translated to public domains so the mongo clients (Compass and mongo shell) want to connect to the primary using it's inter-cluster domain name which makes it unaccessible from outside of the cluster.

inscapist commented 3 years ago

Mind sharing your deployment specs?

dgdevops commented 3 years ago

Sure @sagittaros, here it is:

apiVersion: mongodbcommunity.mongodb.com/v1
kind: MongoDBCommunity
metadata:
  name: rs-name
  namespace: mongodb
spec:
  additionalMongodConfig:
    net.ssl.allowInvalidCertificates: true
    net.ssl.mode: preferTLS
    storage.wiredTiger.engineConfig.journalCompressor: zlib
  members: 3
  persistent: true
  replicaSetHorizons:
  - horizon: cg0.public-domain:27017
  - horizon: cg1.public-domain:27017
  - horizon: cg2.public-domain:27017
  security:
    authentication:
      modes:
      - SCRAM
    roles:
    - db: admin
      privileges:
      - actions:
        - listCollections
        resource:
          collection: ""
          db: ""
      role: customMonitoring
      roles: []
    tls:
      caConfigMapRef:
        name: mongodb-cg-rs-cert
      certificateKeySecretRef:
        name: mongodb-cg-rs-cert
      enabled: true
  statefulSet:
    spec:
      template:
        spec:
          containers:
          - env:
            - name: KUBE_NAMESPACE
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.namespace
            - name: KUBE_LABEL_hostname
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.name
            - name: CHECK_INTERVAL
              value: "10"
            image: <OMITTED>:202104081338
            name: mongo-<OMITTED>
            resources:
              limits:
                cpu: 200m
                memory: 200M
              requests:
                cpu: 150m
                memory: 150M
          - args:
            - --mongodb.uri=mongodb://admin:<OMITTED>@127.0.0.1:27017/admin?ssl=true&sslinsecure=true
            image: eu.gcr.io/mgmt-238514/public/bitnami/mongodb-exporter:0.11.0
            name: mongodb-exporter
            resources:
              limits:
                cpu: 150m
                memory: 150M
              port: 9216
              requests:
                cpu: 100m
                memory: 100M
          imagePullSecrets:
          - name: gcr-registry
          initContainers:
          - name: mongod-posthook
            resources:
              limits:
                cpu: 200m
                memory: 300M
              requests:
                cpu: 100m
                memory: 200M
          - name: mongodb-agent-readinessprobe
            resources:
              limits:
                cpu: 200m
                memory: 300M
              requests:
                cpu: 100m
                memory: 200M
      volumeClaimTemplates:
      - metadata:
          name: data-volume
        spec:
          accessModes:
          - ReadWriteOnce
          resources:
            requests:
              storage: 30Gi
          storageClassName: persistent-ssd
  type: ReplicaSet
  users:
  - db: admin
    name: admin
    passwordSecretRef:
      key: ""
      name: mongodb-cg-admin-secret
    roles:
    - db: admin
      name: clusterAdmin
    - db: admin
      name: userAdminAnyDatabase
    - db: admin
      name: readWriteAnyDatabase
    scramCredentialsSecretName: mongodb-cg-admin
  version: 4.2.6
inscapist commented 3 years ago

I just tried the same connection string format on my machine.

mongo \
    mongodb://USER:PASS@rs.DOMAIN:30000,rs.DOMAIN:30001,rs.DOMAIN:30002/?replicaSet=my-mongodb \
    --tls

I have to add --tls to make it work.

Also, I normally use mongo-shell, which is stricter in root CA check (I recall Compass behave more similarly to mongo-shell than mongo). You have to install your CA cert for Compass to work.

mongosh \
    mongodb://USER:PASS@rs.DOMAIN:30000,rs.DOMAIN:30001,rs.DOMAIN:30002/?replicaSet=my-mongodb \
    --tls \
    --tlsCAFile=/home/ubuntu/.local/share/mkcert/rootCA.pem
dgdevops commented 3 years ago

Thanks @sagittaros for checking, I posted the deployment manifest above. I added the --tls flag, getting the same errors.

inscapist commented 3 years ago

@dgdevops the deployment spec looks good to me except:

      caConfigMapRef:
        name: mongodb-cg-rs-cert
      certificateKeySecretRef:
        name: mongodb-cg-rs-cert

caConfigMapRef should be a CA cert, not to be confused with the server/mongo certificate.

I also suggest you try connecting using mongosh:

mongosh \
    mongodb://cg0.public-domain:27017,cg1.public-domain:27017,cg2.public-domain:27017/?replicaSet=rs-name \
    --tls \
    --tlsCAFile=/path/to/rootCA.pem
dgdevops commented 3 years ago

@sagittaros , I am using Kubernetes cert-manager to issue the certificates and the caConfigMapRef pointing to mongodb-cg-rs-cert is generated from the mongodb-cg-rs-cert Kubernetes secret's ca.crt field that is created by Let's encrypt.

priyolahiri commented 3 years ago

New doc added: docs/external_access.md and samples in config/samples/external_access

dgdevops commented 3 years ago

Hello @sagittaros, Many thanks for the suggestions, eventually I made it work. @priyolahiri. thank you for the documentation.

inscapist commented 3 years ago

Great! What was the issue? @dgdevops

dgdevops commented 3 years ago

Hey @sagittaros, I am not entirely sure, the only difference I can see between the working & not working scenarios is that I redeployed the replicaset with the ReplicaSetHorizons feature enabled from scratch. Previously I added the ReplicaSetHorizons parameter to the existing replicaset.