aio-libs / aiohttp

Asynchronous HTTP client/server framework for asyncio and Python
https://docs.aiohttp.org
Other
14.96k stars 1.99k forks source link

Server Side Event client disconnects after 5 minutes #4770

Open goliatone opened 4 years ago

goliatone commented 4 years ago

🐞 Describe the bug

I'm using aiosseclient to receive Sever Side Events. After 5 minutes the client timeouts. If you look at the source code you see the library starts a ClientSession with default params (timeout 5 minutes)

async with aiohttp.ClientSession() as session:
        response = await session.get(url, **kwargs)

I want to submit a PR with a fix. I tried to set a ClientTimeout with total=None like so:

 timeout = aiohttp.ClientTimeout(total=None, connect=None,
                      sock_connect=None, sock_read=None)
    async with aiohttp.ClientSession(timeout=timeout) as session:
        response = await session.get(url, **kwargs)

Now, after 5 minutes I get an iohttp.ServerDisconnectedError. Connecting with a JS client from the browser for periods > 5min seems to indicate server is ok.

💡 To Reproduce Follow sample quart SSE server here

💡 Expected behavior ClientSession should run indefinitely.

📋 Logs/tracebacks

📋 Your version of the Python

$ python --version
Python 3.8.3
...

📋 Your version of the aiohttp/yarl/multidict distributions

$ python -m pip show aiohttp
Name: aiohttp
Version: 3.6.2
Summary: Async http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author: Nikolay Kim
Author-email: fafhrd91@gmail.com
License: Apache 2
Location: /usr/local/lib/python3.8/site-packages
Requires: multidict, async-timeout, chardet, attrs, yarl
Required-by: aiosseclient
$ python -m pip show multidict
Name: multidict
Version: 4.7.6
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /usr/local/lib/python3.8/site-packages
Requires: 
Required-by: yarl, aiohttp
$ python -m pip show yarl
Name: yarl
Version: 1.4.2
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl/
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /usr/local/lib/python3.8/site-packages
Requires: idna, multidict
Required-by: aiohttp

📋 Additional context

cjrh commented 4 years ago

You didn't provide a reproducer; for future reference, if you supply a reproducer that can be copy-pasted and run, so that a maintainer can test it immediately makes it much, much more likely that the issue will be looked at. I had to do a bit of digging to create files to test this.

Firstly, I can't reproduce the problem. The timeout parameter client-side appears to work fine. Using a total timeout parameter of None also works fine (no timeout is set). I note that you got a "Server disconnected" error message. That tells me the server, i.e. quart, dropped the connection in that instance, not the client.

Secondly, you don't have to provide a fix to aiosseclient because the kwargs are passed through to internal session.get calls, which means you can set the client timeout by providing a timeout kwargs to the call to aiosseclient itself.

I tested on Windows but os shouldn't make any difference here.

venv

$ pip freeze
aiofiles==0.5.0
aiohttp==3.6.2
aiosseclient==0.0.1
async-timeout==3.0.1
attrs==19.3.0
blinker==1.4
chardet==3.0.4
click==7.1.2
h11==0.9.0
h2==3.2.0
hpack==3.0.0
Hypercorn==0.9.5
hyperframe==5.2.0
idna==2.9
itsdangerous==1.1.0
Jinja2==2.11.2
MarkupSafe==1.1.1
multidict==4.7.6
priority==1.3.0
Quart==0.12.0
toml==0.10.1
typing-extensions==3.7.4.2
Werkzeug==1.0.1
wsproto==0.15.0
yarl==1.4.2

client.py

import asyncio

import aiohttp
from aiosseclient import aiosseclient

async def main():
    async for event in aiosseclient(
        "http://127.0.0.1:5000/sse", timeout=aiohttp.ClientTimeout(total=None),
    ):
        print(event, end=", ", flush=True)

asyncio.run(main())

server.py

import asyncio
import itertools
from typing import Optional

from quart import Quart, make_response

app = Quart(__name__)

class ServerSentEvent:
    def __init__(
        self,
        data: str,
        *,
        event: Optional[str] = None,
        id: Optional[int] = None,
        retry: Optional[int] = None,
    ) -> None:
        self.data = data
        self.event = event
        self.id = id
        self.retry = retry

    def encode(self) -> bytes:
        message = f"data: {self.data}"
        if self.event is not None:
            message = f"{message}\nevent: {self.event}"
        if self.id is not None:
            message = f"{message}\nid: {self.id}"
        if self.retry is not None:
            message = f"{message}\nretry: {self.retry}"
        message = f"{message}\r\n\r\n"
        return message.encode("utf-8")

@app.route("/sse")
async def sse():
    async def send_events():
        print("starting loop")
        for i in itertools.count():
            yield ServerSentEvent(f"{i}").encode()
            if i % 10 == 0:
                print(i, end=", ", flush=True)

            await asyncio.sleep(1.0)

    response = await make_response(
        send_events(),
        {
            "Content-Type": "text/event-stream",
            "Cache-Control": "no-cache",
            "Transfer-Encoding": "chunked",
        },
    )
    response.timeout = None  # No timeout for this route
    return response

app.run()

client output:

$ python client.py
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24,
25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47
, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69,
70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92
, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111,
112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129,
130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147,
148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165,
166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183,
184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201,
202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219,
220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237,
238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255,
256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273,
274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291,
292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307, 308, 309,
310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327,
328, 329, 330, 331, 332, 333, 334, 335, 336, 337, 338, 339, 340, 341, 342, 343, 344, 345,
346, 347, 348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363,
364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377, 378, 379, 380, 381,
382, 383, 384, 385, 386, 387, 388, 389, 390, 391, 392, 393, 394, 395, 396, 397, 398, 399,
400, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417,
418, 419, 420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 431, 432, 433, 434, 435,
436, 437, 438, 439, 440, 441, 442, 443, 444, 445, 446, 447, 448, 449,
<snip>

server output:

$ python server.py
[2020-05-30 14:03:19,633] Running on 127.0.0.1:5000 over http (CTRL + C to quit)
starting loop
0, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 20
0, 210, 220, 230, 240, 250, 260, 270, 280, 290, 300, 310, 320, 330, 340, 350, 360, 370, 380
, 390, 400, 410, 420, 430, 440, 450, 460, 470,
<snip>
cjrh commented 4 years ago

(Oh and I forgot to mention, Python 3.8.3)