This fixes #100 by returning the slug from the URL only if the request was successful. If request wasn't successful, we wait for some time and retry up to 5 times. The wait time is the same as the one used while getting pages in the scraper as it would give a good cooldown period before continuing.
This fixes #100 by returning the slug from the URL only if the request was successful. If request wasn't successful, we wait for some time and retry up to 5 times. The wait time is the same as the one used while getting pages in the scraper as it would give a good cooldown period before continuing.