crashvb / docker-registry-client-async

An AIOHTTP based Python REST client for the Docker Registry.
Apache License 2.0
5 stars 6 forks source link

No route to host error with IPv6 #25

Closed skrech closed 2 years ago

skrech commented 2 years ago

Hello crashvb,

Thanks for the great package, I'm happy that there is docker/oci registry client that works with python!

I've encountered a problem, though. aiodns package, used for DNS resolution in DRCA, has a shortcoming of returning either IPv4 or IPv6 addresses, but not both on DNS resolution. This causes domains having both A and AAAA records to be resolved by aiodns with only IPv6. Since IPv6 is not common yet, most OSes have working IPv6 stacks but no default route setup, which leads to OSError No Route to Host when trying get a blob, for example. This is especially true if the application using DRCA is running in Docker container, however, then the error is even more cryptic: instead of No Route to Host, it fails with Errno 99 [Cannot assign requested address].

For this exact problem, aiohttp switched the default resolver to ThreadedResolver from AsyncResolver (https://github.com/aio-libs/aiohttp/commit/9fbb7d708375e0ba01d435c1e8cf41912381c0fc).

The problem with IPv6 is coming from c-ares C lib, actually: https://github.com/c-ares/c-ares/issues/70

And it's tracked in 'aiodns', here: https://github.com/saghul/aiodns/issues/23

One workaround is to pair AsyncResolver with aiohttp.TCPConnector class that have an explicit family kwargs set to IPv4. Unfortunately, DRCA only supports passing custom arguments to AsyncResolver, but not to aiohttp.TCPConnector. So, the following hack should be made:

class DRCA_PATCHED(drca.DockerRegistryClientAsync):
    async def _get_client_session(self):
        if not self.client_session:
            self.client_session = aiohttp.ClientSession(
                connector=aiohttp.TCPConnector(
                    resolver=aiohttp.AsyncResolver(**self.resolver_kwargs),
                    ssl=self.ssl,
                    family=socket.AF_INET,
                )
            )
        return self.client_session

So, I guess, this will be encountered very often by users (and even more often when domains start to acquire AAAA records, I've encountered it with trying to download layers of python:3.9 image from docker.io), so maybe it will be better for DRCA to allow passing Connector options, or be able to switch the Resolver class.

crashvb commented 2 years ago

@skrech,

Good find! :)

This might explain some of the weird DNS behavior I was experienced in downstream projects on-prem. There were several edge cases with c-ares not handling server rejections, process hosts aliases, or honoring lookup parameters ... but it got to the point where I gave up and did DNS resolution out-of-band up a layer (sad-but-true).

I agree that it should be possible to both specify the resolver and have the TCPConnector parameters exposed; I'll push out a fix.

Thank you

crashvb commented 2 years ago

I pushed a fix to the dev branch; let me know if you think I'm missing something ...

crashvb commented 2 years ago

Released in v0.2.2.

skrech commented 2 years ago

Thank you for the fast reaction! Changes look good! :)