Open hydrosquall opened 8 years ago
Consider making a scrapy spider rather than using asyncio / requests/ bs4 combo to visit the 24k+ athlete bios. It can then be deployed to scrapinghub as long as it will complete within 24 hrs.
Consider making a scrapy spider rather than using asyncio / requests/ bs4 combo to visit the 24k+ athlete bios. It can then be deployed to scrapinghub as long as it will complete within 24 hrs.