Error 'AIOBoto3ResourceFactory._create_autoload_property.' was never awaited on Object.copy_from

[ekapratama93]

• Async AWS SDK for Python version: 8.0.5
• Python version: 3.8
• Operating System: Ubuntu 20.04

Description

I tried to update tags using Object.copy_from but got error sys:1: RuntimeWarning: coroutine 'AIOBoto3ResourceFactory._create_autoload_property.' was never awaited. Here is the error message from the code.

Traceback (most recent call last):
  File "s3.py", line 36, in <module>
    loop.run_until_complete(main())
  File "/home/eka/.pyenv/versions/3.8.5/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "s3.py", line 29, in main
    await obj.copy_from(CopySource={'Bucket':'mybucket', 'Key':file.key}, TaggingDirective='REPLACE', MetadataDirective='REPLACE', Tagging="duplicate=true", Metadata=obj.metadata)
  File "/home/eka/.local/share/virtualenvs/s3-VfuamJbd/lib/python3.8/site-packages/aioboto3/resources/factory.py", line 239, in do_action
    response = await action(self, *args, **kwargs)
  File "/home/eka/.local/share/virtualenvs/s3-VfuamJbd/lib/python3.8/site-packages/aioboto3/resources/action.py", line 41, in __call__
    response = await getattr(parent.meta.client, operation_name)(**params)
  File "/home/eka/.local/share/virtualenvs/s3-VfuamJbd/lib/python3.8/site-packages/aiobotocore/client.py", line 97, in _make_api_call
    request_dict = await self._convert_to_request_dict(
  File "/home/eka/.local/share/virtualenvs/s3-VfuamJbd/lib/python3.8/site-packages/aiobotocore/client.py", line 145, in _convert_to_request_dict
    request_dict = self._serializer.serialize_to_request(
  File "/home/eka/.local/share/virtualenvs/s3-VfuamJbd/lib/python3.8/site-packages/botocore/validate.py", line 297, in serialize_to_request
    raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid type for parameter Metadata, value: <coroutine object AIOBoto3ResourceFactory._create_autoload_property.<locals>.property_loader at 0x7ff1e33ffc40>, type: <class 'coroutine'>, valid types: <class 'dict'>
sys:1: RuntimeWarning: coroutine 'AIOBoto3ResourceFactory._create_autoload_property.' was never awaited

What I Did

async def main():
    async with boto3.resource('s3') as s3:
        audio_bucket = await s3.Bucket('mybucket')

        async for file in audio_bucket.objects.all():
            obj = await audio_bucket.Object(key=file.key)
            await obj.copy_from(CopySource={'Bucket':'mybucket', 'Key':file.key}, TaggingDirective='REPLACE', MetadataDirective='REPLACE', Tagging="a=1", Metadata=obj.metadata)

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

[ekapratama93]

Here are my dependencies:

aioboto3==8.0.5
aiobotocore==1.0.4
aiohttp==3.6.2
boto3==1.12.32
botocore==1.15.32
s3transfer==0.3.3

[terricain]

Sorry I've taken so long to get back to this.

Its to do with how boto3 creates properties on the higher level service objects. As you call obj.metadata it'll go and load the metadata for that object synchronously. Not really applicable to async so you need to await obj.metadata as it returns a co-routine

so this snippet seems to work for me

import asyncio
import aioboto3

async def main():
    async with aioboto3.resource('s3') as s3:
        audio_bucket = await s3.Bucket('aioboto3-test-bucket')

        async for file in audio_bucket.objects.all():
            obj = await audio_bucket.Object(key=file.key)
            obj_metadata = await obj.metadata
            await obj.copy_from(CopySource={'Bucket':'aioboto3-test-bucket', 'Key':file.key}, TaggingDirective='REPLACE', MetadataDirective='REPLACE', Tagging="a=1", Metadata=obj_metadata)

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

Feel free to reopen this if your still having issues.