search

thumbor-community / aws

Thumbor AWS extensions
MIT License
155 stars 70 forks source link

Use aiobotocore 2.2.0 to support assume role credentials #157

Open gcavalcante8808 opened 2 years ago

gcavalcante8808 commented 2 years ago

Scenario

This PR bumps aiobotocore to its latest version (2.2.0), allowing to authenticate using IAM Roles/WebIdentity credentials with thumbor-aws.

What has been Done

gcavalcante8808 commented 2 years ago

Relates to: https://github.com/thumbor-community/aws/issues/156

Bladrak commented 1 year ago

@gcavalcante8808 hi, did you try this on a live server? Did it handle the load properly?

gcavalcante8808 commented 1 year ago

@Bladrak Well, as I ended up leaving the company I was working for, I can only talk about the period I was there: it ran for 6 months even in production without complications =D

Bladrak commented 1 year ago

Ok great :) Would you mind switching the target branch to master and rebasing this? I think we will be able to merge it now!

Bladrak commented 1 year ago

@gcavalcante8808 there seems to be an issue with Circle CI from your fork. It may be related to https://circleci.com/docs/oss/#build-pull-requests-from-forked-repositories

If a user submits a pull request to your repository from a fork, but no pipeline is triggered, then the user most likely is following a project fork on their personal account rather than the project itself of CircleCI, causing the jobs to trigger under the user’s personal account and not the organization account. To resolve this issue, have the user unfollow their fork of the project on CircleCI and instead follow the source project. This will trigger their jobs to run under the organization when they submit pull requests.

Can you check that this is right for your fork? And update accordingly if need be?

Bladrak commented 1 year ago

Hi @gcavalcante8808 could you check out the CircleCI issue?

oliverschewe commented 9 months ago

Any updates on this? I would also like to use assumeRole via AWS Web Identity Token and it looks like old boto version does not support it.

I am currently getting follow errors.

Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/botocore/credentials.py", line 502, in _protected_refresh metadata = self._refresh_using() File "/usr/local/lib/python3.9/site-packages/botocore/credentials.py", line 643, in fetch_credentials return self._get_cached_credentials() File "/usr/local/lib/python3.9/site-packages/botocore/credentials.py", line 654, in _get_cached_credentials self._write_to_cache(response) File "/usr/local/lib/python3.9/site-packages/botocore/credentials.py", line 679, in _write_to_cache self._cache[self._cache_key] = deepcopy(response) File "/usr/local/lib/python3.9/copy.py", line 161, in deepcopy rv = reductor(4) TypeError: cannot pickle 'coroutine' object 2024-01-24 17:09:42 thumbor:ERROR [BaseHander.execute_image_operations] cannot pickle 'coroutine' object Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/thumbor/handlers/__init__.py", line 146, in execute_image_operations result = await self.context.modules.result_storage.get() File "/usr/local/lib/python3.9/site-packages/tc_aws/result_storages/s3_storage.py", line 55, in get key = await super(Storage, self).get(path) File "/usr/local/lib/python3.9/site-packages/tc_aws/aws/storage.py", line 53, in get return await self.storage.get(file_abspath) File "/usr/local/lib/python3.9/site-packages/tc_aws/aws/bucket.py", line 75, in get return await self._client.get_object( File "/usr/local/lib/python3.9/site-packages/aiobotocore/client.py", line 76, in _make_api_call http, parsed_response = await self._make_request( File "/usr/local/lib/python3.9/site-packages/aiobotocore/client.py", line 96, in _make_request return await self._endpoint.make_request(operation_model, request_dict) File "/usr/local/lib/python3.9/site-packages/aiobotocore/endpoint.py", line 68, in _send_request request = self.create_request(request_dict, operation_model) File "/usr/local/lib/python3.9/site-packages/botocore/endpoint.py", line 115, in create_request self._event_emitter.emit(event_name, request=request, File "/usr/local/lib/python3.9/site-packages/botocore/hooks.py", line 356, in emit return self._emitter.emit(aliased_event_name, **kwargs) File "/usr/local/lib/python3.9/site-packages/botocore/hooks.py", line 228, in emit return self._emit(event_name, kwargs)

Bladrak commented 9 months ago

Hi @oliverschewe it seems this PR is a bit outdated. If you'd like to retake the PR and submit an updated one, I'll be happy to review.