psf / requests

A simple, yet elegant, HTTP library.
https://requests.readthedocs.io/en/latest/
Apache License 2.0
52.2k stars 9.34k forks source link

一个while True requests运行的爬虫程序,会报错:HTTPSConnectionPool(host='***', port=443): Max retries exceeded with url: /aa (Caused by SSLError(OSError(24, 'Too many open files'))) #6766

Closed wucyAAA closed 4 months ago

wucyAAA commented 4 months ago

with requests.session() as session: while True: try: task = get_task() if task is None: time.sleep(1) continue response = session.get(url, headers=headers, proxies=task['ip'],timeout=6) parse_detail(response .text) except Exception as e: report_data(task['url'], [], task['tid'],False,str(e)) logger.error(e) time.sleep(1) 大概就是这样的结构,只有request操作,是什么地方没写好会造成这样的错误呢?能给点建议吗? 小白第一次提问,可能说的不够清晰,抱歉抱歉

github-actions[bot] commented 4 months ago

As described in the template, we won't be able to answer questions on this issue tracker. Please use Stack Overflow