oras-project / oras-py

ORAS Python SDK
https://oras-project.github.io/oras-py/
Apache License 2.0
40 stars 36 forks source link

Update headers in do_request method, provider.py #129

Closed my5cents closed 8 months ago

my5cents commented 8 months ago

I work with remote registry, available via http URL. I set basic auth and tried to do some operations within ORAS. I was able to get tags, manifest and make pull, but push operation failed. Debugging showed that during the push operation POST and PUT methods loose provided auth headers. It is crucial for remote registry, since further re-auth logic tries to reach registry locally. Below is my log and truncated stack trace (it's too long):

2024-04-03 18:19:09.422       oras_manager:40  INFO: ['0.0.1', '0.1.1', 'latest']
2024-04-03 18:19:11.818             logger:246 INFO: Retrying in 3 seconds - error: HTTPConnectionPool(host='docker-registry-realm', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x00000168BBEDA960>: Failed to resolve 'docker-registry-realm' ([Errno 11001] getaddrinfo failed)"))
2024-04-03 18:19:15.878             logger:246 INFO: Retrying in 5 seconds - error: HTTPConnectionPool(host='docker-registry-realm', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x00000168BBED93D0>: Failed to resolve 'docker-registry-realm' ([Errno 11001] getaddrinfo failed)"))
2024-04-03 18:19:21.954             logger:246 INFO: Retrying in 11 seconds - error: HTTPConnectionPool(host='docker-registry-realm', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x00000168BBEDAED0>: Failed to resolve 'docker-registry-realm' ([Errno 11001] getaddrinfo failed)"))
2024-04-03 18:19:34.029             logger:246 INFO: Retrying in 29 seconds - error: HTTPConnectionPool(host='docker-registry-realm', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x00000168BBEDB950>: Failed to resolve 'docker-registry-realm' ([Errno 11001] getaddrinfo failed)"))
2024-04-03 18:20:04.108             logger:246 INFO: Retrying in 83 seconds - error: HTTPConnectionPool(host='docker-registry-realm', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x00000168BBF28290>: Failed to resolve 'docker-registry-realm' ([Errno 11001] getaddrinfo failed)"))
FAILED                                                                   [100%]Retrying in 3 seconds - error: HTTPConnectionPool(host='docker-registry-realm', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x00000168BBEDA960>: Failed to resolve 'docker-registry-realm' ([Errno 11001] getaddrinfo failed)"))
Retrying in 5 seconds - error: HTTPConnectionPool(host='docker-registry-realm', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x00000168BBED93D0>: Failed to resolve 'docker-registry-realm' ([Errno 11001] getaddrinfo failed)"))
Retrying in 11 seconds - error: HTTPConnectionPool(host='docker-registry-realm', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x00000168BBEDAED0>: Failed to resolve 'docker-registry-realm' ([Errno 11001] getaddrinfo failed)"))
Retrying in 29 seconds - error: HTTPConnectionPool(host='docker-registry-realm', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x00000168BBEDB950>: Failed to resolve 'docker-registry-realm' ([Errno 11001] getaddrinfo failed)"))
Retrying in 83 seconds - error: HTTPConnectionPool(host='docker-registry-realm', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x00000168BBF28290>: Failed to resolve 'docker-registry-realm' ([Errno 11001] getaddrinfo failed)"))

.....

tests\smoke_tests\test_registry_smoke.py:32: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
libs\oras\oras_manager.py:47: in __init__
    res = registry.get_manifest('busybox')
..\..\AppData\Local\Programs\Python\Python312\Lib\site-packages\oras\decorator.py:35: in __call__
    return self.func(cls, *args, **kwargs)
..\..\AppData\Local\Programs\Python\Python312\Lib\site-packages\oras\provider.py:864: in get_manifest
    response = self.do_request(get_manifest, "GET", headers=headers)
..\..\AppData\Local\Programs\Python\Python312\Lib\site-packages\oras\decorator.py:60: in __call__
    return self.func(cls, *args, **kwargs)
..\..\AppData\Local\Programs\Python\Python312\Lib\site-packages\oras\provider.py:914: in do_request
    if self.authenticate_request(response):
..\..\AppData\Local\Programs\Python\Python312\Lib\site-packages\oras\provider.py:1004: in authenticate_request
    authResponse = self.session.get(h.realm, headers=headers, params=params)  # type: ignore
..\..\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\sessions.py:602: in get
    return self.request("GET", url, **kwargs)
..\..\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\sessions.py:589: in request
    resp = self.send(prep, **send_kwargs)
..\..\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\sessions.py:703: in send
    r = adapter.send(request, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <requests.adapters.HTTPAdapter object at 0x00000168BBED9250>
request = <PreparedRequest [GET]>, stream = False
timeout = Timeout(connect=None, read=None, total=None), verify = True
cert = None, proxies = OrderedDict()

    def send(
        self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None
    ):

.....

>           raise ConnectionError(e, request=request)
E           requests.exceptions.ConnectionError: HTTPConnectionPool(host='docker-registry-realm', port=80): Max retries exceeded with url: / (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x00000168BBF28BC0>: Failed to resolve 'docker-registry-realm' ([Errno 11001] getaddrinfo failed)"))

..\..\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\adapters.py:519: ConnectionError

Process finished with exit code 1

If do_request method is updated to include provided 'Authorization' header, push works fine as well as other operations:

2024-04-03 18:26:18.777       oras_manager:40  INFO: ['0.0.1', '0.1.1', 'latest']
2024-04-03 18:26:18.866             logger:252 DEBUG: Preparing layer {'mediaType': 'application/vnd.oci.image.layer.v1.tar+gzip', 'size': 2939, 'digest': 'sha256:1532aa4a0fb892d0091e7791d31e438f344d4b62dec6f8cb5f21afd8eee05849', 'annotations': {'org.opencontainers.image.title': 'busybox'}}
2024-04-03 18:26:19.443             logger:252 DEBUG: Preparing config {'mediaType': 'application/vnd.unknown.config.v1+json', 'size': 2, 'digest': 'sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a'}
2024-04-03 18:26:20.086       oras_manager:50  INFO: Push response: <Response [201]>
2024-04-03 18:26:20.252             logger:246 INFO: Successfully pulled C:\Users\my5cents\AppData\Local\Temp\oras-tmp.i0s7_bfz\busybox.
2024-04-03 18:26:20.252       oras_manager:54  INFO: Pull response: ['C:\\Users\\my5cents\\AppData\\Local\\Temp\\oras-tmp.i0s7_bfz\\busybox']
PASSED                                                                   [100%]Preparing layer {'mediaType': 'application/vnd.oci.image.layer.v1.tar+gzip', 'size': 2939, 'digest': 'sha256:1532aa4a0fb892d0091e7791d31e438f344d4b62dec6f8cb5f21afd8eee05849', 'annotations': {'org.opencontainers.image.title': 'busybox'}}
Preparing config {'mediaType': 'application/vnd.unknown.config.v1+json', 'size': 2, 'digest': 'sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a'}
Successfully pushed my.remote.registry.domain.edu/busybox:latest
Successfully pulled C:\Users\my5cents\AppData\Local\Temp\oras-tmp.i0s7_bfz\busybox.

Process finished with exit code 0
vsoch commented 8 months ago

This looks good - you are exposing this variable to more functions. Could you please bump the version (in version.py) and add a line to CHANGELOG.md? For the formatting/linting errors you can see the issue in the GitHub action output and run the same versions (e.g., black) locally to fix.

my5cents commented 8 months ago

Could you please bump the version (in version.py) and add a line to CHANGELOG.md? For the formatting/linting errors you can see the issue in the GitHub action output and run the same versions (e.g., black) locally to fix.

Done

vsoch commented 8 months ago

@my5cents looks like you just need one more run of black and we're good (take note of the version).

yurnov commented 8 months ago

I reformatted remaining part, it should be fine