DNS resolution fail on Kubernetes

[mDistefanoFacileIt]

Hello folks, with my team we are trying to use AMPHP to execute a series of concurrency tasks. Specifically, each task deals with:

• reading values from a redis cache
• if the values are not in the cache, executes one or more requests to APIs provided by our partners
• records the values in the cache

To do this, we installed amphp/amp (^3.0), amphp/http-client (^5.1), amphp/redis (^2.0), league/uri (7.4.1) as dependency. Our project is based on Symfony and run PHP 8.1 We build our Amp\Futures and execute them with Amp\Future\awaitAll().

Locally we use docker-compose to execute our docker images and everything works as we expect. In the test and production environment we use K8S with the same docker images used locally. Running the tests, however, we get an error when running requests, specifically when amp/dns tries to resolve the DNS, this is regardless of the configured uri. The error we get is as follows:

Connection to 'host.api.com:443' failed

Amp\Socket\ConnectException: Connection to tcp://host.api.com:443 failed after 3 attempts; previous attempts: DNS resolution for host.api.com failed: Name resolution failed for 'host.api.com.svc.cluster.local'; server returned error code: 3 (NXDomain), DNS resolution for host.api.com failed: Name resolution failed for 'host.api.com.svc.cluster.local'; server returned error code: 3 (NXDomain) in vendor/amphp/socket/src/RetrySocketConnector.php:50

These are systems outside of our network. The requests we want to execute are constructed quite simply:

        $request = new Request(
            'https://host.api.com/example',
            'POST',
            {'foo' : 'bar'},
        );

Reading the .svc.cluster.local it appears to be trying to resolve DNS internally within our K8S cluster. We also tried to configure the URI as https://host.api.com. (see the dot at the end) to avoid DNS resolution as reported in https://mrkaran.dev/posts/ndots-kubernetes/ . But then we encounter some certificate validation problems:

Amp\Socket\TlsException: TLS negotiation failed: stream_socket_enable_crypto(): Peer certificate CN=`host.api.com' did not match expected CN=`host.api.com.' 

We then performed a further test, creating a script that would simply run Amp\Dns\resolve() on the uri we were interested in. By simply running this function in K8S env, the DNS are resolved correctly. We think it's somenthing between \Amp\Http\Client\Connection\DefaultConnectionFactory and \Amp\Dns\Rfc1035StubDnsResolver::resolve to cause the error but cannot undestand what it is. Do you have any suggestions as to whether we are doing it wrong or there is some problem with the resolution of the DNS under K8S?

We have already read https://github.com/amphp/dns/issues/99 without success

[mDistefanoFacileIt]

We figured out how to fix this.

Yuo can use ndots to avoid DNS resolution configuring the request as (see the dot at the end of https://host.api.com):

        $request = new Request(
            'https://host.api.com./example',
            'POST',
            {'foo' : 'bar'},
        );

Then you can configure the TLS handshake through HTTClientBuilder as:

        $tlsContext = (new ClientTlsContext())->withPeerName('host.api.com');

        $connectContext = (new ConnectContext())->withTlsContext($tlsContext);
        $factory = new DefaultConnectionFactory(null, $connectContext);
        $pool = ConnectionLimitingPool::byAuthority(3, $factory);

        return (new HttpClientBuilder())
            ->usingPool($pool)
            ->build();

[trowski]

Please have a look at your ndots configuration. From the looks of the error message you posted, it appears that the resolver is attempting to look up names prepended with other search names, and running out of attempts.

https://www.reddit.com/r/kubernetes/comments/duj86x/comment/f76k3hc/

Perhaps we're not handling ndots > 1 correctly and changes should be made here.

[mDistefanoFacileIt]

Hello @trowski in our resolve.conf we have:

options ndots:5