search

spyoungtech / grequests

Requests + Gevent = <3
https://pypi.python.org/pypi/grequests
BSD 2-Clause "Simplified" License
4.49k stars 331 forks source link

Error from Gevent: cannot switch to another thread #44

Closed ychen306 closed 8 years ago

ychen306 commented 10 years ago

Hi. I am using grequests to send a bunch of requests. After about 2000 requests, error below is reported. error: cannot switch to a different thread (libev) error creating signal/async pipe: Too many open files Any idea what happened? I use imap with default pool size.

rtdean commented 10 years ago

You've exhausted your per-process open file descriptor limit. You can see this limit at ulimit -n.

Are you using stream=True on your requests? Are you holding onto the yielded response object past when you need it? Something is holding fd's open, and that's why you are experiencing this issue.