Making a request which has a very large payload (e.g. 50MB) usually receives a 413 error from the server it is calling. This means raise_for_status() returns a ClientResponseError as expected, which can be handled by downstream code.
This works as expected with a simple await session.post(...), but inside an async with session.post(...) block, attempts to raise the status error cause a ClientOSError with a broken pipe.
To Reproduce
import json
import sys
from aiohttp import ClientSession
from aiohttp.client_exceptions import ClientResponseError
async with ClientSession(base_url='https://www.google.com') as sess:
large_data = [
*[[0.000000,0.000000 + i * 0.00001] for i in range(0, 10000000)],
]
payload = {"data": large_data}
assert sys.getsizeof(json.dumps(payload)) > 1024*500
sess = ClientSession(base_url='https://www.google.com')
# This works
try:
res = await sess.request('post', '/zsdfj', json=payload)
res.raise_for_status()
except ClientResponseError:
print("Request failed")
# This fails with a BrokenPipe
try:
async with sess.request('post', '/zsdfj', json=payload) as res:
res.raise_for_status()
except ClientResponseError:
print("Request in context manager failed")
Expected behavior
Both requests should result in a ClientReponseError.
Logs/tracebacks
Traceback (most recent call last):
File "/opt/conda/envs/py38/lib/python3.8/site-packages/aiohttp/client_reqrep.py", line 644, in write_bytes
await writer.write_eof()
File "/opt/conda/envs/py38/lib/python3.8/site-packages/aiohttp/http_writer.py", line 158, in write_eof
await self.drain()
File "/opt/conda/envs/py38/lib/python3.8/site-packages/aiohttp/http_writer.py", line 171, in drain
await self._protocol._drain_helper()
File "/opt/conda/envs/py38/lib/python3.8/site-packages/aiohttp/base_protocol.py", line 90, in _drain_helper
await asyncio.shield(waiter)
aiohttp.client_exceptions.ClientOSError: [Errno 32] Broken pipe
---------------------------------------------------------------------------
ClientOSError Traceback (most recent call last)
Cell In[22], line 24
22 try:
23 async with sess.request('post', '/zsdfj', json=payload) as res:
---> 24 res.raise_for_status()
25 except ClientResponseError:
26 print("Request in context manager failed")
File /opt/conda/envs/py38/lib/python3.8/site-packages/aiohttp/client.py:1213, in _RequestContextManager.__aexit__(self, exc_type, exc, tb)
1201 async def __aexit__(
1202 self,
1203 exc_type: Optional[Type[BaseException]],
(...)
1210 # explicitly. Otherwise connection error handling should kick in
1211 # and close/recycle the connection as required.
1212 self._resp.release()
-> 1213 await self._resp.wait_for_close()
File /opt/conda/envs/py38/lib/python3.8/site-packages/aiohttp/client_reqrep.py:1094, in ClientResponse.wait_for_close(self)
1092 async def wait_for_close(self) -> None:
1093 if self._writer is not None:
-> 1094 await self._writer
1095 self.release()
File /opt/conda/envs/py38/lib/python3.8/site-packages/aiohttp/client_reqrep.py:644, in ClientRequest.write_bytes(self, writer, conn)
642 protocol.set_exception(new_exc)
643 except asyncio.CancelledError:
--> 644 await writer.write_eof()
645 except Exception as exc:
646 protocol.set_exception(exc)
File /opt/conda/envs/py38/lib/python3.8/site-packages/aiohttp/http_writer.py:158, in StreamWriter.write_eof(self, chunk)
155 if chunk:
156 self._write(chunk)
--> 158 await self.drain()
160 self._eof = True
File /opt/conda/envs/py38/lib/python3.8/site-packages/aiohttp/http_writer.py:171, in StreamWriter.drain(self)
163 """Flush the write buffer.
164
165 The intended use is to write
(...)
168 await w.drain()
169 """
170 if self._protocol.transport is not None:
--> 171 await self._protocol._drain_helper()
File /opt/conda/envs/py38/lib/python3.8/site-packages/aiohttp/base_protocol.py:90, in BaseProtocol._drain_helper(self)
88 waiter = self._loop.create_future()
89 self._drain_waiter = waiter
---> 90 await asyncio.shield(waiter)
ClientOSError: [Errno 32] Broken pipe
I think the server has disconnected prematurely, causing that exception to happen while trying to close the connection cleanly. Might have to try and suppress the exception there in __aexit__()...
Describe the bug
Making a request which has a very large payload (e.g. 50MB) usually receives a 413 error from the server it is calling. This means
raise_for_status()
returns aClientResponseError
as expected, which can be handled by downstream code. This works as expected with a simpleawait session.post(...)
, but inside anasync with session.post(...)
block, attempts to raise the status error cause aClientOSError
with a broken pipe.To Reproduce
Expected behavior
Both requests should result in a
ClientReponseError
.Logs/tracebacks
Python Version
aiohttp Version
multidict Version
yarl Version
OS
Related component
Client
Additional context
No response
Code of Conduct