nismod / ukpopulation

Population and demographics projection module, developed for ITRC/MISTRAL
MIT License
13 stars 7 forks source link

Fixed forbidden url bug when downloading 2016 household projections #70

Closed ld-archer closed 2 years ago

ld-archer commented 2 years ago

When attempting to download the zip file for 2016 household projections from nrscotland (link), the code previously downloaded a 403 error page and cached that in place of the 2016-house-proj-detailed-counc-princ.zip file. This caused downstream errors in the microsimulation package as it was expecting a zip file, but encountering a html file in disguise.

I defined a user-agent based (below) on linux and firefox to pass to the requests.get function which seems to fix the problem.

headers = {'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:105.0) Gecko/20100101 Firefox/105.0'}
response = requests.get(scotland_src, headers=headers)