Download fails for downloadfGFW

[Jo-Schie]

HI @goergen95 .

Did you accidently overwrite something again with the co2 layer download functions. I am currently unable to download GFW data so I cannot tst #5 .

devtools::install_github("mapme-initiative/mapme.forest")
library(sf)
library(mapme.forest)

## TEST DOWNLOAD
# read in polygons of interest
aoi = st_read(system.file("extdata", "aoi_polys.gpkg", package = "mapme.forest"))

# download GFW data for the area of interest
raster_files = downloadfGFW(shape = aoi,
                            basename = "pkgTest",
                            dataset = "GFC-2019-v1.7",
                            outdir = "shared/xyz/data/",
                            keepTmpFiles = T)

gives me

trying URL 'https://storage.googleapis.com/earthenginepartners-hansen/GFC-2019-v1.7/Hansen_GFC-2019-v1.7_treecover2000_20N_100E.tif'
Content type 'application/octet-stream' length 518834343 bytes (494.8 MB)
==================================================
downloaded 494.8 MB

trying URL 'https://storage.googleapis.com/earthenginepartners-hansen/GFC-2019-v1.7/Hansen_GFC-2019-v1.7_lossyear_20N_100E.tif'
Content type 'image/tiff' length 91984705 bytes (87.7 MB)
==================================================
downloaded 87.7 MB

trying URL 'http://gfw2-data.s3.amazonaws.com/climate/Hansen_emissions/2018_loss/per_pixel/20N_100E_tCO2_pixel_AGB_masked_by_loss.tif'
Error in download.file(urls[i], localname) : 
  cannot open URL 'http://gfw2-data.s3.amazonaws.com/climate/Hansen_emissions/2018_loss/per_pixel/20N_100E_tCO2_pixel_AGB_masked_by_loss.tif'
In addition: Warning messages:
1: In if (!is.na(co2tiles)) { :
  the condition has length > 1 and only the first element will be used
2: In download.file(urls[i], localname) :
  cannot open URL 'http://gfw2-data.s3.amazonaws.com/climate/Hansen_emissions/2018_loss/per_pixel/20N_100E_tCO2_pixel_AGB_masked_by_loss.tif': HTTP status was '403 Forbidden'

So I guess he is correctly downloading treecover and lossyear layer from google but then again fails with the amazonws download.

Also i noticed that the function is now called downloadfGFW instead of downloadGFW. Not sure if that is on purpose.

[goergen95]

Looks like a new error to me. Seems like we haven't properly fixed the download function the last time. But its not related to the changes in #5. I will look into it shortly.

[Jo-Schie]

Cool. Aren't the masks also available here? Not sure it it is the same but if we can avoid the AMAZON storage then maybe this facilitates things...

Data mask (datamask) Three values representing areas of no data (0), mapped land surface (1), and permanent water bodies (2).

Or is it again the CO2 problem? Then maybe we should exclude it completely for the moment and add it as a new feature request for the future...?

[goergen95]

That turned out to be an really interesting bug. It was caused that the csv file from where we extract the locations of available CO2-emissions tiles were again query able from the server. However, the links to the GTiffs still are pointing to a dead end causing then an error in the download.file() and the function to fail.

When looking into it I discovered that there were fairly new data sets published on the GFW Data Portal, e.g. one layer of Forest Greenhouse Gas Emissions which closely resembles the data we used before. This new data set now actually is available globally while the Biomass Loss layer before was only available for the tropics. The layer is part of the publication of Harris et al (2021). Maybe have a look into it, because there seem to be a number of new interesting layers (e.g. Forest Carbon Removals.

For now, I established a fairly simple routine to download this new layer in 20aad3f. I am thinking, that an API package to the GlobalForestWatch Data API would be quite useful because there are a bunch of interesting data sets. If none such R package exist we could think about implementing that in either {mapme.forest} or a new package?

Also, please note that the routines analysing this layer are in need of some adaptation. The layer represents greenhouse gas emission between 2000 and 2020. Thus we need to first divide it by 20 to get an information about the yearly average and then aggregate on the user specified year vector. So, the current commit ensures functionality but I would advise to currently not use the CO2 layer in analysis.

EDIT: Added refernce to the commit with the mentioned changes.

[Jo-Schie]

ok download issue seems to be fixed for now.