bluegreen-labs / daymetr

An R Interface to the Daymet Web Services
http://bluegreen-labs.github.io/daymetr/
Other
30 stars 17 forks source link

2021 data now available #50

Closed jessicaguo closed 3 years ago

jessicaguo commented 3 years ago

2021 DAYMET data through June is now available: https://daac.ornl.gov/cgi-bin/dsviewer.pl?ds_id=1904

Does the download_daymet() function need to be updated to access this data? When I use 'end = 2021' as an argument, I get the following error: End year exceeds valid data range!

Thanks for providing this great tool!

khufkens commented 3 years ago

The half yearly release cycle is new to me but if you want recent data you should set the end year to 2021 and force = TRUE, this then ignores the check which assumes there is not data for the current year. However, I have a hard time reaching the server. You might want to try when it is back up.

download_daymet(site = "Oak Ridge National Laboratories",
                lat = 36.0133,
                lon = -84.2625,
                start = 1980,
                end = 2021,
                force = TRUE,
                internal = TRUE)
jessicaguo commented 3 years ago

The server is working for me, but even with end = 2021 and force = TRUE, the last date returned is 12/30/2020. This appears to be the case for several sites I checked. Why might this be so?

khufkens commented 3 years ago

The product your are looking at is provisional and doesn't make it into the API until later. If you want access you will need to bulk download the data (netCDF files) and subset them yourself I'm afraid. This isn't difficult, but does require juggling some files etc.

dlebauer commented 3 years ago

@khufkens the data are on a thredds server here: https://thredds.daac.ornl.gov/thredds/catalog/ornldaac/1904/catalog.html

Maybe throwing darts, but would replacing 1860 with 1904 on this line~~ https://github.com/bluegreen-labs/daymetr/blob/a37dafa0f5715ae77fbf1b3cd8d7e904f09b832a/R/zzz.r#L14 ~~magically give access to the monthly latency data?

Update

looking a little bit more I think I see the issue - the data are publicly available through this API but not from the single-pixel api https://daymet.ornl.gov/single-pixel/api/data

used by the download_daymet function

dlebauer commented 3 years ago

@jessicaguo in the meantime this might be helpful for extracting pixels from tiles https://github.com/ornldaac/thredds_opendap_r_max_temperature/blob/master/opendap_r_v1.Rmd

khufkens commented 3 years ago

Hi @dlebauer,

You are correct, the reasons why it isn't in the package is that it isn't exposed through the normal API.

The solution you linked to works, but I'm a bit hesitant to include it as this is a bit of a hack. Upstream ORNL changes can really mess with the package stability (unit checks) which ultimately trickle down to CRAN compliance and me having to address these issues (in short notice). Since this is all done on free time I tend to limit potential overhead like this.

I'll ask Michele at ORNL if there is an option to include these products. This might be the best solution.

Cheers, K

khufkens commented 3 years ago

I got word back from ORNL and they are slowly moving to integrate the product. So I won't put in the effort to work on a workaround. For now the reference to the ORNL github code should do for most. I'll pin the item should the question arise again.