search

niclasmattsson / GlobalEnergyGIS

Generates input data for energy models on renewable energy in arbitrary world regions using public datasets. Written in Julia 1.x.
MIT License
64 stars 24 forks source link

Error while `download_datasets()` for WDPA #11

Open euronion opened 3 years ago

euronion commented 3 years ago

Calling

download_datasets()

fails when trying to download the WDPA data:

Downloading dataset 4: WDPA (protected areas):
Downloading to <folder>/WDPA.zip...
ERROR: LoadError: HTTP.ExceptionRequest.StatusError(404, "GET", "/current/WDPA_WDOECM_May2021_Public_feadb1b4f30799a6dc3ad0b16116d3530ec4a477898f4e10e097e2030e167128_shp.zip", HTTP.Messages.Response:
"""
HTTP/1.1 404 Not Found
Content-Type: application/xml
Transfer-Encoding: chunked
Connection: keep-alive
Date: Mon, 09 Aug 2021 10:39:42 GMT
Server: AmazonS3
X-Cache: Error from cloudfront
Via: 1.1 ef13dd533b8dc9dcfdc35449cf88f808.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: FRA50-C1
X-Amz-Cf-Id: ej2uT3JopzgNoMWx1fJcAyHYKVl4vUDwiwaAeg3E91Spjh74C6BWxw==

""")

The link for the WDPA does not work (I tried opening it manually) and seems to be a link individually created for each download.

siminh86 commented 3 years ago

hello, I got the same error and do not know how can I cope with it:

Downloading dataset 4: WDPA (protected areas): Downloading to C:/Users/simhaj-local/OneDrive - ltu.se/Documents/syntathic_data\WDPA.zip... ERROR: HTTP.ExceptionRequest.StatusError(404, "GET", "/current/WDPA_WDOECM_May2021_Public_feadb1b4f30799a6dc3ad0b16116d3530ec4a477898f4e10e097e2030e167128_shp.zip", HTTP.Messages.Response: """ HTTP/1.1 404 Not Found Content-Type: application/xml Transfer-Encoding: chunked Connection: keep-alive Date: Sat, 11 Sep 2021 15:16:05 GMT Server: AmazonS3 X-Cache: Error from cloudfront Via: 1.1 8021f954d329869476f935f2fb14e66e.cloudfront.net (CloudFront) X-Amz-Cf-Pop: ARN1-C1 X-Amz-Cf-Id: 49toKQigKXOBwZVnsc0fmxrXfO6FR2ECyB4cf61gCxecqnJbjKk2yQ==

""")

goldenamir commented 3 years ago

Issue solved no worries :)

janohlen commented 3 years ago

Hello,

The issues seems to reappear in October 2021.

Downloading dataset 4: WDPA (protected areas):
Downloading to D:/GISdata\WDPA.zip...

Hardcoded WDPA url no longer working, probably due to an end-of-month update at www.protectedplanet.net.
Retrying using a guessed url for the current month...
Downloading to D:/GISdata\WDPA.zip...
ERROR: HTTP.ExceptionRequest.StatusError(404, "GET", "/current/WDPA_WDOECM_Oct2021_Public_all_shp.zip", HTTP.Messages.Response:
"""
HTTP/1.1 404 Not Found
Content-Type: application/xml
Transfer-Encoding: chunked
Connection: keep-alive
Date: Sun, 10 Oct 2021 08:58:25 GMT
Server: AmazonS3
X-Edge-Origin-Shield-Skipped: 0
X-Cache: Error from cloudfront
Via: 1.1 784dd167d622737126ee2d76985e7d3c.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: FRA2-C2
X-Amz-Cf-Id: cQhkpCJcxxFpGy3PS4ym52R0zczY2zJzj6QpqHI3_1nEERab3T3gyg==
""")

It appears that protectedplanet.net has changed the filename structure from https://d1gam3xoknrgr2.cloudfront.net/current/WDPA_WDOECM_Oct2021_Public_all_shp.zip to https://d1gam3xoknrgr2.cloudfront.net/current/WDPA_Oct2021_Public_shp.zip

Therefore the guessing of URL function fails.

Further, the naming convention in the zip file has also changed from WDPA-shapefile_$i to WDPA_Oct2021_Public_shp_$i"

This change also affects the rasterization of the protected areas in make_auxiliary_datasets.jl.