[BUG] Timeout issues in stream mode

[ZentixUA]

Describe the bug Some hosts have an issue where, during streaming, timeout seems to have no effect; closing the connection on certain sources can take 20 or even 30 seconds, even when no new data is coming through. Other sources can completely hang and even with a timeout setting of (30, 60 * 10), they remain open for hours without closing.

To Reproduce

import asyncio
import json
import sys
import uuid

from curl_cffi.requests import AsyncSession

url = "https://huggingface.co/chat/conversation"
data = {
    "model": "mistralai/Mistral-7B-Instruct-v0.2",
    "preprompt": ""
}
chat_data = {
    'inputs': 'Hey! Say a few words',
    'id': str(uuid.uuid4()),
    'response_id': str(uuid.uuid4()),
    'is_retry': False,
    'web_search': False
}

async def main():
    async with AsyncSession() as session:
        response = await session.request(
            'POST',
            url,
            json=data,
            impersonate="chrome120",
            default_headers=False
        )
        conv = response.json()['conversationId']
        async with session.stream(
                'POST',
                f'{url}/{conv}',
                json=chat_data,
                cookies=response.cookies
        ) as ai_response:
            async for line in ai_response.aiter_lines():
                line = json.loads(line)
                if line.get('type') == 'finalAnswer':
                    print('Final answer, breaking...')
                    break
                print(line)

if __name__ == '__main__':
    if sys.platform == 'win32':
        asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
    asyncio.run(main())

I can provide another example, but it is quite private so perker can contact me in Telegram :)

Expected behavior The connection closes immediately

Versions

• OS: Widnows 11
• curl_cffi version: 0.6.0b7
• pip freeze dump:

  aiogTTS==1.1.1
  aiohttp==3.9.1
  aiosignal==1.3.1
  annotated-types==0.6.0
  anyio==3.7.1
  attrs==23.1.0
  beautifulsoup4==4.12.2
  certifi==2023.11.17
  cffi==1.16.0
  charset-normalizer==3.3.2
  click==8.1.7
  colorama==0.4.6
  curl-cffi==0.6.0b7
  dalle3==0.1.0
  distro==1.8.0
  fastapi==0.105.0
  frozenlist==1.4.1
  h11==0.14.0
  httpcore==1.0.2
  httpx==0.25.2
  idna==3.6
  multidict==6.0.4
  numpy==1.26.2
  openai==1.7.2
  orjson==3.9.10
  outcome==1.3.0.post0
  pandas==2.1.4
  pillow==10.2.0
  psutil==5.9.7
  pycparser==2.21
  pydantic==2.5.2
  pydantic_core==2.14.5
  PySocks==1.7.1
  python-dateutil==2.8.2
  pytz==2023.3.post1
  redis==5.0.1
  requests==2.31.0
  requests-toolbelt==1.0.0
  selenium==4.16.0
  setuptools==69.0.3
  six==1.16.0
  sniffio==1.3.0
  sortedcontainers==2.4.0
  soupsieve==2.5
  starlette==0.27.0
  sydney-py==0.20.0
  tenacity==8.2.3
  tqdm==4.66.1
  trio==0.23.2
  trio-websocket==0.11.1
  typing_extensions==4.9.0
  tzdata==2023.3
  undetected-chromedriver @ git+https://www.github.com/ultrafunkamsterdam/undetected-chromedriver@783b8393157b578e19e85b04d300fe06efeef653
  urllib3==2.1.0
  uvicorn==0.24.0.post1
  validators==0.22.0
  websockets==11.0.3
  wsproto==1.2.0
  yarl==1.9.4

  Additional context

  • Which session are you using? async or sync? - Async
  • If using async session, which loop implementation are you using? - asyncio.WindowsSelectorEventLoopPolicy(), but the same thing happens for linux (ubuntu)

[coletdjnz]

For streaming, the timeout is currently only a connection timeout and not a read timeout, as per https://github.com/yifeikong/curl_cffi/issues/156.

You can achieve a read timeout by setting this before you request:

 session.curl.setopt(CurlOpt.LOW_SPEED_LIMIT, 1)  # 1 byte per second
 session.curl.setopt(CurlOpt.LOW_SPEED_TIME, math.ceil(20.0)) # timeout after 20 seconds of receiving <=1byte/s

[ZentixUA]

session.acurl.setopt(CurlOpt.LOW_SPEED_LIMIT, 1)

TypeError: initializer for ctype 'void *' must be a cdata pointer, not int

[perklet]

acurl is not a curl instance, asyncio session uses a pool of curl instances, you can do like this:

session = AsyncSession(..., 
    curl_options={
        CurlOpt.LOW_SPEED_LIMIT: 1, 
        CurlOpt.LOW_SPEED_TIME: math.ceil(20.0)
    }
)

[ZentixUA]

Thank you. What about a break in the middle of iterating content as in the example? In aiohttp, the Response closes immediately, for instance. But with curl_cffi, I'm waiting 20-30 seconds. :(

[perklet]

Sorry, libcurl does not allow us to close the connection early. You can break from the iteration loop, but the connection will be floating around anyway. If your request number is not very big, it might be acceptable to just let the connections be closed automatically.

[perklet]

For streaming, the timeout is currently only a connection timeout and not a read timeout, as per #156.

You can achieve a read timeout by setting this before you request:

 session.curl.setopt(CurlOpt.LOW_SPEED_LIMIT, 1)  # 1 byte per second
 session.curl.setopt(CurlOpt.LOW_SPEED_TIME, math.ceil(20.0)) # timeout after 20 seconds of receiving <=1byte/s

Change this to default since v0.6.0b9.