Describe the bug
When I use get_object my code starts to run very slowly. Many open aiohttp connections begin to be created. These connections cannot be released. The length of the aiohttp.BaseConnector._acquired set grows to the size of the connection pool and does not fall back.
Checklist
[x] I have reproduced in environment where pip check passes without errors
[x] I have provided pip freeze results
[x] I have provided sample code or detailed way to reproduce
[x] I have tried the same code in botocore to ensure this is an aiobotocore specific issue
[x] I have tried similar code in aiohttp to ensure this is is an aiobotocore specific issue
[x] I have checked the latest and older versions of aiobotocore/aiohttp/python to see if this is a regression / injection
When I use only generate_presigned_url/list_objects_v2 - everything ok. Code working fast, no exceptions/timeouts
10 calls of _get_objects func withoutget_object call:
bash-4.2# ss -s
Total: 12 (kernel 1802)
TCP: 116 (estab 3, closed 110, orphaned 61, synrecv 0, timewait 11/0), ports 0
Transport Total IP IPv6
* 1802 - -
RAW 0 0 0
UDP 1 1 0
TCP 6 5 1
INET 7 6 1
FRAG 0 0 0
when I start using the get_object the problems start
10 calls of _get_objects func withget_object call:
bash-4.2# ss -s
Total: 27 (kernel 1809)
TCP: 137 (estab 18, closed 116, orphaned 66, synrecv 0, timewait 12/0), ports 0
Transport Total IP IPv6
* 1809 - -
RAW 0 0 0
UDP 1 1 0
TCP 21 20 1
INET 22 21 1
FRAG 0 0 0
evertthing appears to be good. But aiobotocore tests start failng :)
Also, if i take a ClientSession from AIOHTTPSession class of aiobotocore and try to send a request to https;//... by this session, it doesnt work too. It will be just infinity request.
Describe the bug When I use
get_object
my code starts to run very slowly. Many open aiohttp connections begin to be created. These connections cannot be released. The length of the aiohttp.BaseConnector._acquired set grows to the size of the connection pool and does not fall back.Checklist
pip check
passes without errorspip freeze
resultspip freeze results
Environment:
Code(connection):
Code(use case)
When I use only
generate_presigned_url
/list_objects_v2
- everything ok. Code working fast, no exceptions/timeouts 10 calls of _get_objects func withoutget_object
call:when I start using the
get_object
the problems start 10 calls of _get_objects func withget_object
call:I found out that if i place
response.release()
here https://github.com/aio-libs/aiobotocore/blob/master/aiobotocore/httpsession.py#L227evertthing appears to be good. But aiobotocore tests start failng :)
Also, if i take a ClientSession from AIOHTTPSession class of aiobotocore and try to send a request to https;//... by this session, it doesnt work too. It will be just infinity request.