Open alanhdu opened 8 years ago
Asyncio is Python's new standard asynchronous I/O library. We should use it when scraping (we'll have to replace requests with aiohttp).
We may want to upgrade to Python 3.5 so we can get the async and await syntactic sugar.
async
await
An alternative would be to use a threadpool with concurrent.futures. Look into the facebook scraper.
concurrent.futures
Asyncio is Python's new standard asynchronous I/O library. We should use it when scraping (we'll have to replace requests with aiohttp).
We may want to upgrade to Python 3.5 so we can get the
async
andawait
syntactic sugar.