[Feature]: Allow to configure mongodb credentials separately from URI

[matthias-pichler]

Is there an existing issue for this?

• [X] I have searched the existing issues

Summary

When running an external MongoDB cluster connectivity is configured via the APPSMITH_MONGODB_URI environment variable, which in the docs always contains the username and password as well, eg:

APPSMITH_MONGODB_URI=mongodb+srv://<USERNAME>:<PASSWORD>@<MONGO.HOST.NAME>/<DATABASENAME>

now during startup of the instance I can see that the env variables:

APPSMITH_MONGODB_USER=....
APPSMITH_MONGODB_PASSWORD....

also get populated. However when I configure the username and password via these variables the check_replicate_set fails with an "Authorization Failure". Presumably because the client in the script is only initialized with the URI and not the two variables above.

https://github.com/appsmithorg/appsmith/blob/1d1f01389c5a2aa65cf1823dd722183095d5cbfd/deploy/docker/utils/bin/check_replica_set.js#L5-L8

Why should this be worked on?

This is a problem when deploying appsmith to ECS on AWS because secrets can be configured to be read from Amazon SecretsManager during startup, however the mongodb secrets only contain the values (username, password, hostname) individually but not as a constructed URI.

This only leaves me with the possibility to configure the URI as a regular environment variable instead of a secret which exposes the credentials in the AWS UI and makes it impossible to apply additional access control to the secrets. It also causes problems when using automatic rotation as the values aren't read dynamically anymore.

[sharat87]

Hey @matthias-pichler-warrify, thank you for the details and for bringing this to our attention.

Is it possible to use the full MongoDB URI itself as a secret on AWS?

[matthias-pichler]

So when creating the DocumentDB cluster a secret in secretsmanager is created with roughly following fields:

{
  "dbClusterIdentifier":"xxxxxxx-xxxxxxx",
  "password":".....",
  "engine":"mongo",
  "port":27017,
  "host":"xxxx-xxxxx.cluster-xxx.eu-west-1.docdb.amazonaws.com",
  "ssl":true,
  "username":"appsmith"
}

so no the entire URI is not in the secret itself. The thing with ECS secrets is that you can only use entire secrets or individual fields, but you can't concat any strings. The only way to do that would be to set the URI as a regular env var and construct it in the CloudFormation Template. But this is the exact problem I described, because it won't work with rotation and show the value in the AWS Management UI.

[sharat87]

@matthias-pichler-warrify, got it, thanks for the details.

One, I get the usecase we need to solve by accepting a different way to specify the MongoDB credentials, this we'll pick up soon. I'll keep this issue for this problem since the title already accurately reflects this.

Two, unfortunately, Appsmith doesn't support DocumentDB at all today. It needs MongoDB v5+, and that's what it is tested to work. DocumentDB compatibility is in the roadmap, but it's in very early stages right now, we're figuring out how far away we are, and how much needs to be fixed/changes to be compatible. Until then, I'd recommend using a MongoDB cluster instead.

[matthias-pichler]

@sharat87 thanks for the information. What features that appsmith uses aren't supported in DocumentDB? https://docs.aws.amazon.com/documentdb/latest/developerguide/compatibility.html#mongodb-50

[sharat87]

@matthias-pichler-warrify, unfortunately, that's the part we're trying to figure out, and (even sadder) it's not that straightforward. For example, the most recent problem we faced in starting Appsmith server with DocumentDB was that DocumentDB has stricter limits on the lengths of index names. So we had to go back and rename the indexes, in a backwards-compatible way, so it can work on both MongoDB and DocumentDB. Once that is fixed, we are now discovering another discrepency, that we're still yet to look into and figure out.