Docdb connection string error - The connection string contains invalid user information. If the username or password contains a colon (:) or an at-sign (@) then it must be urlencoded

[evainthecloud]

connection string for docdb is passed with secrets manager name in the same format as specified in the README.

mongodb://${secretsname}@hostname:port/?ssl=true&ssl_ca_certs=rds-combined-ca-bundle.pem&replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false

When launching the catalog from Athena - the below error is displayed:

"The connection string contains invalid user information. If the username or password contains a colon (:) or an at-sign (@) then it must be urlencoded".

The username nor the password has a colon or at-sign. Tried different ways of invoking the secretsmanager - it still doesn't work.

[avirtuos]

Can you share the stack-trace of the error? be sure to remove any sensitive information before posting.

[avirtuos]

Also what version of the connector are you using? (we have a template for issues which will show you the questions we typical ask in order to streamline support)

[evainthecloud]

Describe the bug Failed to invoke lambda function due to com.amazonaws.services.lambda.invoke.LambdaFunctionException: The connection string contains invalid user information. If the username or password contains a colon (:) or an at-sign (@) then it must be urlencoded (Service: AmazonAthena; Status Code: 400; Error Code: MetadataException; Request ID:xxxxxxxxxxxxxxxxxxxx; Proxy: null)

To Reproduce Steps to reproduce the behavior:

1. Go to 'Athena’, select the ‘catalog’ created for document db(Assumption – connection to external data source – documentdb is completed through the cloudformation stack setup for the Lambda function to establish the connection)
2. It keeps looping on the screen and the above error message is displayed(under “Describe the bug” section)

Expected behavior The tables should have been listed for the docdb database, once the docdb catalog is selected in Athena.

Screenshots / Exceptions / Errors When you enter the username and password for the docdb in the connection string, it works fine and connects. For security reasons, we want to pass the secrets manager name instead and when we use the secrets manager name, we encounter this error.

CloudWatch Logs Trace: 2021-06-22 19:36:17 INFO DocDBMetadataHandler:129 - getConnStr: No environment variable found for catalog xxxx, using default default_xxxx 2021-06-22 19:36:18 INFO DocDBConnectionFactory:52 - getOrCreateConn: enter 2021-06-22 19:36:18 WARN CompositeHandler:104 - handleRequest: Completed with an exception. java.lang.IllegalArgumentException: The connection string contains invalid user information. If the username or password contains a colon (:) or an at-sign (@) then it must be urlencoded at com.mongodb.ConnectionString.(ConnectionString.java:318) ~[task/:?] at com.mongodb.client.MongoClients.create(MongoClients.java:61) ~[task/:?] at com.amazonaws.athena.connectors.xxxx.DocDBConnectionFactory.getOrCreateConn(DocDBConnectionFactory.java:56) ~[task/:?] at com.amazonaws.athena.connectors.xxxx.DocDBMetadataHandler.getOrCreateConn(DocDBMetadataHandler.java:118) ~[task/:?] at com.amazonaws.athena.connectors.xxxx.DocDBMetadataHandler.doListSchemaNames(DocDBMetadataHandler.java:145) ~[task/:?] at com.amazonaws.athena.connector.lambda.handlers.MetadataHandler.doHandleRequest(MetadataHandler.java:236) ~[task/:?] at com.amazonaws.athena.connector.lambda.handlers.CompositeHandler.handleRequest(CompositeHandler.java:132) ~[task/:?] at com.amazonaws.athena.connector.lambda.handlers.CompositeHandler.handleRequest(CompositeHandler.java:100) [task/:?] at lambdainternal.EventHandlerLoader$2.call(EventHandlerLoader.java:902) [LambdaSandboxJava-1.0.jar:?] at lambdainternal.AWSLambda.startRuntime(AWSLambda.java:340) [LambdaSandboxJava-1.0.jar:?] at lambdainternal.AWSLambda.(AWSLambda.java:63) [LambdaSandboxJava-1.0.jar:?] at java.lang.Class.forName0(Native Method) ~[?:1.8.0_201] at java.lang.Class.forName(Class.java:348) [?:1.8.0_201] at lambdainternal.LambdaRTEntry.main(LambdaRTEntry.java:150) [LambdaJavaRTEntry-1.0.jar:?]

Connector Details (please complete the following information):

• Version: 2021.21.1
• Name: docdb(mongodb)
• Athena Query IDs [if applicable]

Additional context Add any other context about the problem here.

[evainthecloud]

Hi,

Just wanted to check if there is any insight into this connection string issue on connecting Athena with Docdb?

[avirtuos]

Sorry but we haven't been able to look deeply into this yet. Have you tried urlencoding the username and password just in case there is some other character causing issues? : And @ are just two examples of problematic characters.

[evainthecloud]

For security reasons, we are using the secrets manager name instead of passing the username and password directly in the connection string.

Connection string that we are passing:

mongodb://${secretsname}@hostname:port/?ssl=true&ssl_ca_certs=rds-combined-ca-bundle.pem&replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false

Is there a way to pass the username and password of the secrets manager as placeholders in the connection string? For example - ${secrets.username}.

[avirtuos]

I'm sorry but your response didn't answer my question or I may be missunderstanding your response. When using secrets manager the connector essentially does a find replace of that secret with the contents of your secrets manager secret. If the contents of the secret contains special characters you may need to ensure the secret is URL encoded when storing it in Secrets Manager.

As for your follow up question, that is not presently supported. You'd need to have multiple secrets manager secrets to accomplish that today.

[evainthecloud]

I checked, there are no special characters in the username and password for docdb in secrets manager. Is it possible for us to get onto a call, so I can show you the issue?

[et304383]

@avirtuos when creating a secret in secrets manager to connect to document DB, it will create many keys/values such as username, password, host, port, etc.

I looked at the source code under CachableSecretsManager under getSecret and I cannot see how the values are broken out to grab just the that one would expect to replace ${secretname}.

What is the expected format of the secret in secrets manager?

[avirtuos]

There is no expected format the literal contents of the secret is applied. So, for example some folks put the entire connection string into secrets manager. Then just have a {$my_secret_conneciton_string} as their config.

On Wed, Jul 7, 2021, 1:25 PM et304383 @.***> wrote:

@avirtuos https://github.com/avirtuos when creating a secret in secrets manager to connect to document DB, it will create many keys/values such as username, password, host, port, etc.

I looked at the source code under CachableSecretsManager under getSecret and I cannot see how the values are broken out to grab just the username:password that one would expect to replace ${secretname}.

What is the expected format of the secret in secrets manager?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/awslabs/aws-athena-query-federation/issues/446#issuecomment-875788830, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADVEKEWRKOFE6ITN7YX2XNTTWSEWZANCNFSM47ECKHDA .

[et304383]

Wowsers. That REALLY needs to be documented, as the console immediately drives you to the json format, AND with a DocumentDB type secret (that again, most people will use given they're trying to connect to a DocumentDB) there is no opportunity to put the plain text values in as the README suggests given the example:

mongodb://${docdb_instance_1_creds}@myhostname.com:123/?ssl=true&ssl_ca_certs=rds-combined-ca-bundle.pem&replicaSet=rs0

This example implies you should put a plain text item in with the format

<username>:<password>

So that's in no way compatible with the DocumentDB secret type.

[avirtuos]

You can do that too. It only expects a plaintext item, how you use it is up to you. I agree we can put better examples or at least more text explaining the contents of the secret in our existing examples.

On Wed, Jul 7, 2021, 1:40 PM et304383 @.***> wrote:

Wowsers. That REALLY needs to be documented, as the console immediately drives you to the json format, AND with a DocumentDB type secret (that again, most people will use given they're trying to connect to a DocumentDB) there is no opportunity to put the plain text values in as the README suggests given the example:

@.:123/?ssl=true&ssl_ca_certs=rds-combined-ca-bundle.pem&replicaSet=rs0 @.:123/?ssl=true&ssl_ca_certs=rds-combined-ca-bundle.pem&replicaSet=rs0>

This example implies you should put a plain text item in with the format

: So that's in no way compatible with the DocumentDB secret type. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub , or unsubscribe .

[et304383]

@avirtuos I'm not sure what you mean by "you can do that too." That's literally the only thing you can do?

Why isn't this one as smart as the JDBC connector which can point to an actual properly constructed secret from secrets manager with various key/values?

[avirtuos]

We are not prescriptive about the contents of the secret, for example... You can have one secret that is "username:password" or two secrets with username and password respectively or one secret with the entire connection string.

It might be worth us supporting a nested structure for the secret that allow key-value Jason resolution as well. I'll see if I can write up some examples of this to get feedback from you and other interested folks.

On Wed, Jul 7, 2021, 1:48 PM et304383 @.***> wrote:

@avirtuos https://github.com/avirtuos I'm not sure what you mean by "you can do that too." That's literally the only thing you can do?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/awslabs/aws-athena-query-federation/issues/446#issuecomment-875804097, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADVEKEQMTURUBYO7K3P6663TWSHNTANCNFSM47ECKHDA .

[evainthecloud]

Thank you @et304383 for calling this out. I replaced the plaintext in secrets manager with the values of username and password separted by a colon and it worked fine. Athena is now able to connect to DocumentDB.

[mpereira-dev]

This tripped me up as well. Didn't realize you could set the secret as plain text. Thanks @evainthecloud.