search

mikejohnson51 / climateR

An R 📦 for getting point and gridded climate data by AOI
https://mikejohnson51.github.io/climateR/
MIT License
168 stars 40 forks source link

Add CRU TS4 data? #40

Closed urskalbitzer closed 3 years ago

urskalbitzer commented 3 years ago

Is there any chance to include the CRU TS4 data set in the package?

Like TerraClimate, this is a global data set (and TerraClimate is partly based on it). However, for the area where I am working at (Kibale National Park, Uganda), the temperature data from CRU TS4 reflect a bit better our ground data. And this might be true for other areas as well.

The most recent CRU TS data set can be found here, with links to the most recent publication: https://crudata.uea.ac.uk/cru/data/hrg/cru_ts_4.05/

Thanks for considering the addition.

mikejohnson51 commented 3 years ago

I have been looking around for a THREDDS/web version of this data, I found one that holds an older version (TS4.01). Would that be beneficial?

https://iridl.ldeo.columbia.edu/SOURCES/.UEA/.CRU/

Unfortunately we need a web enabled service for climateR to link to,

urskalbitzer commented 3 years ago

Unfortunately, this data set only goes up to 2016 and I am usually working with more recent years as well. But I highly appreciate the offer to adding that one.

I guess these files here don't do it? (I don't know what THREDDS means).

https://crudata.uea.ac.uk/cru/data/hrg/cru_ts_4.05/cruts.2103051243.v4.05/

mikejohnson51 commented 3 years ago

Unfortunately those wont work. THREDDS is a remote access option which allows us to request just a small subset of the data. Without that, we would need to download and cache the data which is a bit out of climateR scope. I will keep my eyes out for a remote access option for this dataset.

urskalbitzer commented 2 years ago

I have seen that the source that you linked above (https://iridl.ldeo.columbia.edu/SOURCES/.UEA/.CRU/) now includes a more recent cruts4 data set (TS4p05), which might even be the most recent one (including years up to 2020). Would you be able to include that in climateR? That would be very helpful for my work.

mikejohnson51 commented 2 years ago

Hi Urs,

I theory yes. Unfortunately LDEO put that dataset behind a password (while others aren't). For example, you'll see the climateR parser can read their PRISM data, but not the CRU data.

x [master*]> p = read_dap_file(URL = "http://iridl.ldeo.columbia.edu/SOURCES/.OSU/.PRISM/.monthly/dods", "prism")
x [master*]> 
x [master*]> p2 = read_dap_file(URL = "http://iridl.ldeo.columbia.edu/SOURCES/.UEA/.CRU/.TS4p05/.monthly/dods", "cru")
syntax error, unexpected WORD_WORD, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
context: <html^><body><h1>401 Unauthorized</h1>User/password is required.</body></html>
Error in RNetCDF::open.nc(URL) : NetCDF: Authorization failure
Run `rlang::last_error()` to see where the error occurred.

I might be able to pass a login to the link but I am not 100% sure of the steps required. I will open this as a new issue and tag you (in a different repo).

Thanks!

Mike

urskalbitzer commented 2 years ago

Hi Mike, Thanks for looking into accessing the Cruts4 data set. I will respond to the issue in the other repository. Urs

urskalbitzer commented 1 year ago

Hi Mike, I just saw that they published a new data set in May 2022 (v4.06; https://crudata.uea.ac.uk/cru/data/hrg/index.htm#current) and they also provide access through a google earth interface (https://crudata.uea.ac.uk/cru/data/hrg/cru_ts_4.06/ge/). Do you think it would be possible to add this new data set to climateR?

Thanks and all the best, Urs