mapme-initiative / mapme.biodiversity

Efficient analysis of spatial biodiversity datasets for global portfolios
https://mapme-initiative.github.io/mapme.biodiversity/dev
GNU General Public License v3.0
33 stars 7 forks source link

Expand a portfolio time coverage without re-downloading all resources? #184

Closed fBedecarrats closed 1 year ago

fBedecarrats commented 1 year ago

Hello there, I wonder what would be the correct procedure to expand a portfolio time coverage. Let's say that I have done:

library(tidyverse)
library(geodata)
library(sf)
library(mapme.biodiversity)

communes_mada <- gadm("MDG", level=4, path = tempdir()) %>%
  st_as_sf() 

androy <- communes_mada %>%
  filter(NAME_4 == "Androy")

aoi <- init_portfolio(androy, years = 2000:2021) %>%
  get_resources("chirps")

Now I would like to expand the years to cover, say, 2000:2023.

How can I do so while minimizing the reload time. Specifying add_resources = FALSE in init_porfolio() would lead to re-download everything, right?

goergen95 commented 1 year ago

Hi! CHIRPS resource currently seems to be unavailable (see here). The correct way is to set add_resources = FALSE. This will ignore any present GPKGs in the resource directory. When you are then running get_resources(), in general, only files that are not already available will be downloaded. In the case of CHIRPS, as we already discussed in an earlier issue, the complete archive is downloaded, irrespective of the temporal extent specified. So running it a second time against the same output directory should not download anything.

fBedecarrats commented 1 year ago

Thank you @goergen95 for the quick response. This is what I remembered from our disucssion but I had a doubt as it did not seem to work as expected. Now I understand that the unexpected behaviour is due to 2 different issues with the source:

  1. lack of responsiveness of the source (from my tests it is still sort of available, but very, very slow).
  2. In this source, CHIRPS updates have been discontinued in Nov. 2021 for COG format, whereas it is still updated for tif formats.

Sorry for not seeing this initially, I'm openning a specific issue to deal with this.