search

nualpaca / api

0 stars 0 forks source link

Deal with HTTPConnectionPool Errors #14

Closed AlJohri closed 10 years ago

AlJohri commented 10 years ago

See: https://github.com/shazow/urllib3/issues/384

AlJohri commented 10 years ago

The thread has pretty much answered all needed questions. We need to start setting block=True and then we can easily use the global facebook_scraper. However, by using this method, it prevents us from running it with or without the delay method. It will only work in celery since we define the global facebook_scraper in worker_init.

Figure out a solution for this and then we stop unpickling facebook_scraper within each greenlet.

AlJohri commented 10 years ago

Fixed with d5315a807dff11c8035db54bfe1e4b63d7da7779