Support for external MongoDB & Redis?

[hexa2k9]

I'm trying to externalise MongoDB & Redis, but it does not seem to be supported (in a proper way) currently.

We're running Dependabot (currently 0.8.2) deployed using the Helm Chart on Google Cloud Kubernetes Engine and would like to externalise Redis on Cloud Memorystore and MongoDB on MongoDB Atlas as those components are in place for us anyway.

Starting with MongoDB I've set mongodb.enabled: false, mongodb.auth.enabled: true and credentials according to the Atlas User. Apart from that I've set the full MongoDB URL (I assume it needs the full URL rather than just the cluster endpoint?) in env.mongodbUrl which is is something like mongodb+srv://<redacted-user>:<redacted-pass>@<redacted-cluster>-pri.asozm.mongodb.net/dependabot?retryWrites=true&w=majority (I'm still not sure of this is the correct format to put into a ConfigMap or if it only needs the protocol and servername).

The Chart however renders the Deployments (Web & Worker) to pull the MONGODB_PASSWORD from the "mongodb" Secret which is only created in case mongodb.enabled is set to true.

A possible Changeset (for MongoDB) might the the following, but I didn't really verify it so I'll leave it as a bare comment to this issue. At first sight it renders fine using helm template though.

diff --git a/charts/dependabot-gitlab/templates/_helpers.tpl b/charts/dependabot-gitlab/templates/_helpers.tpl
index fc3497e..4e69296 100644
--- a/charts/dependabot-gitlab/templates/_helpers.tpl
+++ b/charts/dependabot-gitlab/templates/_helpers.tpl
@@ -82,6 +82,14 @@ Create the name of the service account to use
 {{/*
 Environment config
 */}}
+{{- define "dependabot-gitlab.mongodb-credentials" -}}
+{{- if .Values.mongodb.enabled -}}
+{{- default .Values.mongodb.fullnameOverride }}
+{{- else -}}
+{{- printf "%s-%s" (include "dependabot-gitlab.name" .) "mongodb" }}
+{{- end -}}
+{{- end -}}
+
 {{- define "dependabot-gitlab.database-credentials" -}}
 {{- if .Values.redis.auth.enabled }}
 - name: REDIS_PASSWORD
@@ -94,7 +102,7 @@ Environment config
 - name: MONGODB_PASSWORD
   valueFrom:
     secretKeyRef:
-      name: {{ .Values.mongodb.fullnameOverride }}
+      name: {{ include "dependabot-gitlab.mongodb-credentials" . }}
       key: mongodb-password
 {{- end }}
 {{- end }}
diff --git a/charts/dependabot-gitlab/templates/secrets-mongodb.yaml b/charts/dependabot-gitlab/templates/secrets-mongodb.yaml
new file mode 100644
index 0000000..06e1f6b
--- /dev/null
+++ b/charts/dependabot-gitlab/templates/secrets-mongodb.yaml
@@ -0,0 +1,11 @@
+{{- if .Values.mongodb.enabled -}}
+{{- else -}}
+{{- if .Values.mongodb.auth.enabled -}}
+apiVersion: v1
+kind: Secret
+metadata:
+  name: {{ include "dependabot-gitlab.mongodb-credentials" . }}
+stringData:
+  mongodb-password: {{ .Values.mongodb.auth.password | b64enc | quote }}
+{{- end -}}
+{{- end -}}

[andrcuns]

Thanks for the issue. Indeed, since it is now tied to reading the value from secrets, it's not possible to start the app without it.

Probably need to add these workaround for both mongodb and redis.

[andrcuns]

It should be fixed now. I don't have a proper way testing it end to end, but by the looks of it, templates now looked correct when redis and mongo deployment is disabled.

When providing external mongodb and redis urls, those should be without the credentials as those need to be fetched from separate env variables, I suspect it would not resolve it properly otherwise.

[hexa2k9]

Thank you @andrcuns

I'll look into migrating the bundled MongoDB Instance data to MongoDB Atlas over the course of this week to confirm this works.

[hexa2k9]

I've checked things on another Dependabot Instance. Externalising Redis works just fine by setting env.redisUrl to an Instance including protocol, eg redis://redis.foo.bar.

As for MongoDB I think there's Application Changes required to config/mongoid.yml as per docs: https://docs.mongodb.com/mongoid/current/tutorials/getting-started-rails/#use-mongodb-atlas

Otherwise the Database Healthcheck fails with a message like ERROR: Database healthcheck failed - Host 'mongodb+srv://<redacted-cluster>-pri.asozm.mongodb.net' should not contain protocol. Did you mean to not use an array?

[andrcuns]

Could you try just the host part? The config does not use the full uri option mentioned in the documentation.

[hexa2k9]

I did. That's how I stumbled upon the Link to the Docs.

The Healthcheck then returns: ERROR: Database healthcheck failed - No primary server is available in cluster: #<Cluster topology=Unknown[<redacted>-pri.asozm.mongodb.net:27017] servers=[#<Server address=<redacted>-pri.asozm.mongodb.net:27017 UNKNOWN>]> with timeout=5, LT=0.015

The Folks at MongoDB Atlas rely on mongo+srv as a protocol for Host discovery. They don't publish A records in DNS.

[andrcuns]

It seems that MONGODB_URI option will have to be introduced. I will have to check if mongoid itself handles these options correctly or the configuration will have to be made dynamic and resolved based on what is provided.

[andrcuns]

@hexa2k9 Mongodb Atlas is now supported with the latest chart version. You can provide env.mongoDbUri value if local install is disabled.

[hexa2k9]

Thank you @andrcuns, I will look into migrating the Embedded MongoDB Instance to Cloud Atlas soon.