search

mongodb / mongo-php-library

The Official MongoDB PHP library
https://mongodb.com/docs/php-library/current/
Apache License 2.0
1.6k stars 261 forks source link

Failed to look up SRV record: A temporary error occurred on an authoritative name server #885

Closed maximeof2m closed 2 years ago

maximeof2m commented 2 years ago

Bug Report

I noticed that when we used in our MONGO_HOST_LIST a username and password with specials characters we have such errors :

request.CRITICAL: Uncaught PHP Exception MongoDB\\Driver\\Exception\\ConnectionTimeoutException: \"Failed to look up SRV record \"_mongodb._tcp.**.**.mongodb.net\": A temporary error occurred on an authoritative name server. Try again later.

If we use username and password without special character evrything is okay

Environment

OS => 20.04.2-Ubuntu PHP Version => 7.4.27 Symfony => 5.4

Php.ini mongodb info:

MongoDB support => enabled MongoDB extension version => 1.12.0 MongoDB extension stability => stable libbson bundled version => 1.20.0 libmongoc bundled version => 1.20.0 libmongoc SSL => enabled libmongoc SSL library => OpenSSL libmongoc crypto => enabled libmongoc crypto library => libcrypto libmongoc crypto system profile => disabled libmongoc SASL => disabled libmongoc ICU => enabled libmongoc compression => enabled libmongoc compression snappy => disabled libmongoc compression zlib => enabled libmongoc compression zstd => disabled libmongocrypt bundled version => 1.3.0 libmongocrypt crypto => enabled libmongocrypt crypto library => libcrypto

Directive => Local Value => Master Value mongodb.debug => no value => no value mongodb.mock_service_id => Off => Off

php -i | grep -E 'mongodb|libmongoc|libbson'

/usr/local/etc/php/conf.d/docker-php-ext-mongodb.ini, mongodb libbson bundled version => 1.20.0 libmongoc bundled version => 1.20.0 libmongoc SSL => enabled libmongoc SSL library => OpenSSL libmongoc crypto => enabled libmongoc crypto library => libcrypto libmongoc crypto system profile => disabled libmongoc SASL => disabled libmongoc ICU => enabled libmongoc compression => enabled libmongoc compression snappy => disabled libmongoc compression zlib => enabled libmongoc compression zstd => disabled libmongocrypt bundled version => 1.3.0 libmongocrypt crypto => enabled libmongocrypt crypto library => libcrypto mongodb.debug => no value => no value mongodb.mock_service_id => Off => Off

Composer

"doctrine/mongodb-odm": "^2.2", "doctrine/mongodb-odm-bundle": "^4.2", "mongodb/mongodb": "^1.8",

MongoDB Atlas server

MongoDB => 4.2.17 Cluster => M10 Nodes => 3

Connexion via mongodb+srv://{user}:{pwd}@{clusterName}.mongodb.net

Test Script

Connexion tested with user/pwd with special char => NOK Connexion tested with user/pwd without special char => OK

The probleme seems to be related to how the library parse user and password

jmikola commented 2 years ago

Any special characters in URI components must be encoded according to RFC 3986 (e.g. rawurlencode()). This is noted in both the documentation for MongoDB\Driver\Manager::__construct() and MongoDB\Client::__construct(). It is also discussed in more detail in our cross-driver Connection String specification.

IIRC, the Atlas UI does not include your username and password when presenting you with a connection string to copy/paste and instead leaves placeholders for you to manually add your credentials (configured elsewhere within Atlas). Please confirm that you're properly encoding all parts of the connection string and follow up if the error persists.

maximeof2m commented 2 years ago

Any special characters in URI components must be encoded according to RFC 3986 (e.g. rawurlencode()). This is noted in both the documentation for MongoDB\Driver\Manager::__construct() and MongoDB\Client::__construct(). It is also discussed in more detail in our cross-driver Connection String specification.

According to RFC 3986, username and password contains only unreserved characters such as "-" and "_" . Following the Connection String specification userinfo allowed characters: ( unreserved / pct-encoded / sub-delims / ":" )

jmikola commented 2 years ago

According to RFC 3986, username and password contains only unreserved characters such as "-" and "_" .

I don't see this discussed in the RFC. Can you cite a particular section for that claim?

Appendix A of RFC 3986 defines the grammar for the userinfo component, but there is no defined grammar for individual usernames and passwords. "3.2.1. User Information" and "7.5. Sensitive Information" appear to be the only sections that discuss usernames and passwords.

That said, the Userinfo (optional) section in the Connection String spec is what I'd consider authoritative here, and RFC 3986 is generally referenced therein to discuss encoding of special characters.

Uncaught PHP Exception MongoDB\\Driver\\Exception\\ConnectionTimeoutException: \"Failed to look up SRV record \"_mongodb._tcp.**.**.mongodb.net\": A temporary error occurred on an authoritative name server. Try again later.

If we use username and password without special character everything is okay

That particular error message originates in _mongoc_hstrerror() and is raised when an SRV callback returns a TRY_AGAIN error from whatever DNS resolution API that libmongoc happens to be compiled with. Looking at the other text in the error message, I think the error itself is raised in _mongoc_get_rr_search(). It's not clear to me how the username and password is affecting DNS resolution, as the service string looks valid (e.g. it doesn't appear as if part of the userinfo component was parsed as service).

Since you confirmed that using a username and password without special characters is OK, can you also confirm that applying rawurlencode() to a username and password with special characters also works? If so, that would confirm that your original problem as a lack of character escaping.

maximeof2m commented 2 years ago

I don't see this discussed in the RFC. Can you cite a particular section for that claim?

2.3. Unreserved Characters

Characters that are allowed in a URI but do not have a reserved purpose are called unreserved. These include uppercase and lowercase letters, decimal digits, hyphen, period, underscore, and tilde.

  unreserved  = ALPHA / DIGIT / "-" / "." / "_" / "~"

URIs that differ in the replacement of an unreserved character with its corresponding percent-encoded US-ASCII octet are equivalent: they identify the same resource. However, URI comparison implementations do not always perform normalization prior to comparison (see Section 6). For consistency, percent-encoded octets in the ranges of ALPHA (%41-%5A and %61-%7A), DIGIT (%30-%39), hyphen (%2D), period (%2E), underscore (%5F), or tilde (%7E) should not be created by URI producers and, when found in a URI, should be decoded to their corresponding unreserved characters by URI normalizers.

As mentionned previously I actually use in userinfo :

Those characters are considered as Unreserved Characters if I refered to the RF 3986 documentation "-" and "_" . I use this syntax in a yaml conf file such mongodb+srv://john_doe:foo-bar@{clusterName}.mongodb.net I tried rawurlencode(john_doe:foo-bar); but such character anyway are url friendly so this fonction as no effect on the string.

On other side I generate an another username without "_" and a password without "-" and it works.

So I really don't know what to do without have to use a weak username and password, because refering to documentation "_" and "-" are allowed but does not work properly.

jmikola commented 2 years ago

As mentionned previously I actually use in userinfo :

  • username contains 1 character "_"
    • password contains 1 character "-"

Apologies. I didn't realize you were referring to your actual username and password values when you previously wrote:

According to RFC 3986, username and password contains only unreserved characters such as "-" and "_" .

I had misinterpreted that statement as a reference to some grammar definition of usernames and passwords in the RFC, which only has a userinfo grammar.

Anyway, I should have enough information to attempt a reproduction of the issue you're experiencing. It's quite possible there is a bug in libmongoc's URI parser, but I will follow up when I have a chance to look into this.

jmikola commented 2 years ago

I created an Atlas cluster using a username and password containing an underscore and hyphen, respectively, and could not reproduce any error with SRV resolution, authentication, or executing a command. I've replaced the cluster and host name with "FOO" and "BAR" below, but everything else is unchanged.

The example script below mongodb.debug to get some additional trace information from libmongoc.

<?php

ini_set('mongodb.debug', 'stderr');

$manager = new MongoDB\Driver\Manager('mongodb+srv://phplib_test:gh-885@FOO.BAR.mongodb.net/myFirstDatabase?retryWrites=true&w=majority');

$cursor = $manager->executeCommand('test', new MongoDB\Driver\Command(['ping' => 1]));
var_dump($cursor->toArray()[0]);

ini_set('mongodb.debug', 'off');

echo "\nAll servers:\n";
foreach ($manager->getServers() as $server) {
    printf("%s:%d: latency = %d ms\n", $server->getHost(), $server->getPort(), $server->getLatency());
}

This produced the following abridged output. Since your original error pertained to SRV resolution, I trimmed the logs after confirming that the DNS lookups succeeded (i.e. the _mongoc_get_rr_search calls between initialization of the topology and server descriptions).

[2022-01-06T15:52:04.892351+00:00]     PHONGO: DEBUG   > Connection string: 'mongodb+srv://phplib_test:gh-885@FOO.BAR.mongodb.net/myFirstDatabase?retryWrites=true&w=majority'
[2022-01-06T15:52:04.892418+00:00]     PHONGO: DEBUG   > Creating Manager, phongo-1.12.0[stable] - mongoc-1.20.0(bundled), libbson-1.20.0(bundled), php-8.1.0
[2022-01-06T15:52:04.892433+00:00]     PHONGO: DEBUG   > Setting driver handshake data: { name: 'ext-mongodb:PHP ', version: '1.12.0 ', platform: 'PHP 8.1.0 ' }
[2022-01-06T15:52:04.892468+00:00]     mongoc: TRACE   > ENTRY: mongoc_topology_description_init():78
[2022-01-06T15:52:04.892490+00:00]     mongoc: TRACE   >  EXIT: mongoc_topology_description_init():97
[2022-01-06T15:52:04.892518+00:00]     client: TRACE   > ENTRY: _mongoc_get_rr_search():453
[2022-01-06T15:52:04.915099+00:00]     client: TRACE   >  EXIT: _mongoc_get_rr_search():568
[2022-01-06T15:52:04.915137+00:00]     client: TRACE   > ENTRY: _mongoc_get_rr_search():453
[2022-01-06T15:52:04.937490+00:00]     client: TRACE   >  EXIT: _mongoc_get_rr_search():568
[2022-01-06T15:52:04.937644+00:00]     mongoc: TRACE   > ENTRY: mongoc_server_description_init():123
[2022-01-06T15:52:04.937683+00:00]     mongoc: TRACE   >  EXIT: mongoc_server_description_init():151
[2022-01-06T15:52:04.937716+00:00]     mongoc: TRACE   > ENTRY: mongoc_server_description_init():123
[2022-01-06T15:52:04.937728+00:00]     mongoc: TRACE   >  EXIT: mongoc_server_description_init():151
[2022-01-06T15:52:04.937738+00:00]     mongoc: TRACE   > ENTRY: mongoc_server_description_init():123
[2022-01-06T15:52:04.937749+00:00]     mongoc: TRACE   >  EXIT: mongoc_server_description_init():151
[2022-01-06T15:52:04.937792+00:00]    cluster: TRACE   > ENTRY: mongoc_cluster_init():2636
[2022-01-06T15:52:04.937825+00:00]    cluster: TRACE   >  EXIT: mongoc_cluster_init():2663
...

object(stdClass)#6 (1) {
  ["ok"]=>
  int(1)
}

All servers:
FOO-shard-00-00.BAR.mongodb.net:27017: latency = 13 ms
FOO-shard-00-01.BAR.mongodb.net:27017: latency = 18 ms
FOO-shard-00-02.BAR.mongodb.net:27017: latency = 33 ms

Assuming the original exception is easily reproducible (noting that it did indicate a temporary error), I'd suggest running the above script with your connection string and seeing how it compares to my output.

I tried rawurlencode(john_doe:foo-bar); but such character anyway are url friendly so this function as no effect on the string.

Just to confirm (not sure if this was a typo), but the username and password should be encoded separately. The delimiting colon should not be encoded, as rawurlencode() would convert that to %3A. That said, even if the userinfo component was malformed and included %3A instead of a colon character, I expect you'd see an authentication failure instead of the SRV error you observed. I observed an AuthenticationException in my local testing.

For reference, parsing these particular URI components happens in mongoc_uri_parse_before_slash, which splits the connection string around the @ symbol. The preceding userinfo component is then parsed by mongoc_uri_parse_userpass, where we see a check for a colon character to delimit a username and password. Given that, it doesn't seem likely that a malformed userinfo component would impact SRV resolution unless it included an unescaped @ character that prevented mongoc_uri_parse_before_slash from parsing the host component.