pedro-ricardo
commented
1 year ago Reproduction Steps
With aiobocotore
This code get stuck on the put_object giving lots of timeouts until it breaks.
import asyncio
from aiobotocore.session import get_session
async def go():
session = get_session()
async with session.create_client('s3',
endpoint_url=URL,
region_name=REGION,
aws_secret_access_key=SECRET_ACCESS_KEY,
aws_access_key_id=ACCESS_KEY_ID) as client:
# Get Object
response = await client.get_object( Bucket='test01', Key='hello.txt')
# Read it
async with response['Body'] as stream:
print(await stream.read())
# Upload Object
resp = await client.put_object(Bucket='test01', Key='new_hello.txt', Body=b"Hello World\n")
print(resp)
loop = asyncio.get_event_loop()
loop.run_until_complete(go())With botocore
This code works as expected
from botocore.session import get_session
session = get_session()
client = session.create_client('s3',
endpoint_url=URL,
region_name=REGION,
aws_secret_access_key=SECRET_ACCESS_KEY,
aws_access_key_id=ACCESS_KEY_ID)
# Get Object
response = client.get_object(
Bucket='test01',
Key='hello.txt')
# Read it
with response['Body'] as stream:
print(stream.read())
# Upload Object
resp = client.put_object(
Bucket='test01',
Key='new_hello.txt',
Body=b"Hello World\n")
print(resp)
thehesiod
Describe the bug Hello there, We are testing an S3 compatible storage and we can READ but can not WRITE with
aiobotocorein there. When using theput_objectit gets stuck in a long timeout and retry loop, until it breaks.To narrow it down, we tested the file upload and download in a number of libraries and here is the summary so far:
We plan on using
Kedrowith this S3 storage and it uses thes3fsthat was built on top ofaiobotocore, that's how I got here :grin:I'm sending the codes I used to to try to upload a file both in
botocoreandaiobotocorealong with the output related with that PUT method in DEBUG mode.Checklist
pip checkpasses without errorspip freezeresultspip freeze results
Environment