scraper.get() 404 limbo

cobalt-uoft / uoft-scrapers

Public web scraping scripts for the University of Toronto.

https://pypi.python.org/pypi/uoftscrapers

MIT License

48 stars 14 forks source link

Closed g3wanghc closed 8 years ago

g3wanghc commented 8 years ago

Scraper.get('http://www.asc.utoronto.ca/Hours---Location.htm') limbos after first attempt with 404 response.

g3wanghc commented 8 years ago

Non 200 status_code should increment attempts.

arkon commented 8 years ago

@g3wanghc Is it still happening?

g3wanghc commented 8 years ago

@arkon Fixed it in my branch. Will make a pull request today.

g3wanghc commented 8 years ago

Adding requests.exceptions.InvalidSchema as exception.

g3wanghc commented 8 years ago

This issue should be fixed with pull request #63.