Northgate: follow 301/302 redirects

adrianshort / uk_planning_scraper

A Ruby gem to get planning applications data from UK council websites.

GNU Lesser General Public License v3.0

27 stars 19 forks source link

Northgate: follow 301/302 redirects #39

Open adrianshort opened 5 years ago

adrianshort commented 5 years ago

Currently the Northgate scraper throws a fatal error when getting a 301/302 redirect on the first scrape. We should follow the redirect rather than fail.

This has been an issue with Camden, Islington and Merton, all of which have moved from HTTP to HTTPS.