Sleeps are currently hard-coded. We've already discovered the site sometimes takes a while before all the content is available to retrieve. And marching to a 1000mS beat is monotonous. Let's vary the sleep durations to be more tolerant and also mimic human interactions with the site, to increase the reliability of the scraping process.
Sleeps are currently hard-coded. We've already discovered the site sometimes takes a while before all the content is available to retrieve. And marching to a 1000mS beat is monotonous. Let's vary the sleep durations to be more tolerant and also mimic human interactions with the site, to increase the reliability of the scraping process.