Day calculation incorrect in getGridMET and others

[SarahGoslee-USDA]

In getGridMET, the base date used in

d = define.dates(startDate, endDate, baseDate = "1979-01-01") followed by subtracting 1 from the date.index in

    urls = paste0(g$base, p$call, "_1979_CurrentYear_CONUS.nc?",
        p$description, "[", min(d$date.index) - 1, ":1:", max(d$date.index) -
            1, "]", g$lat.call, g$lon.call, "#fillmismatch")

results in a malformed url when trying to obtain the beginning of the dataset, and returns dates that are consistently off by one day in later time series.

This is not observed with getPRISM, which doesn't specify a base date, but does appear with other get functions.

Here is a reproducible example comparing the results of getGridMET() and data extracted from a downloaded NetCDF file in the working directory.

library(AOI)
library(climateR)
library(sf)
library(raster)

# create an example point
idpt <- data.frame(lon = -96.89999276, lat = 31.51999451)
idpt <- st_as_sf(x = idpt, coords = c("lon", "lat"), crs = 4269)

# extract maximum temperature from a downloaded file
gridmet1979tmax <- stack("tmmx_1979.nc")
gridmet1979tmax.pt <- extract(gridmet1979tmax, idpt)

# try to download the first day of the available time series
getGridMET(idpt, "tmax", startDate = "1979-01-01", endDate = NULL)

# creates malformed URL with -1 in time fields
# http://thredds.northwestknowledge.net:8080/thredds/dodsC/agg_met_tmmx_1979_CurrentYear_CONUS.nc?daily_maximum_temperature[-1:1:-1][429:1:429][669:1:669]#fillmismatch

# does not return data
#   source    lat       lon       date
# 1 gridmet 31.525 -96.89167 1979-01-01

# try to download the second day of the available time series
getGridMET(idpt, "tmax", startDate = "1979-01-02", endDate = NULL)

# creates a functional URL that downloads the previous day
# http://thredds.northwestknowledge.net:8080/thredds/dodsC/agg_met_tmmx_1979_CurrentYear_CONUS.nc?daily_maximum_temperature[0:1:0][429:1:429][669:1:669]#fillmismatch

# this says it is the date requested, but the tmax for 1979-01-01
#   source    lat       lon       date  tmax
# 1 gridmet 31.525 -96.89167 1979-01-02 268.8

# jan 1 and 2 from the downloaded file
gridmet1979tmax.pt[1, 1:2]

# X28854 X28855
# 268.8  272.4

# all dates are off by one, because of the extra subtraction

[mikejohnson51]

Hi @sgoslee thank you for this! I really appreciate your time in checking this. You also found the error perfectly. I had moved the -1 indexing to define.dates and forget to remove it from the getGridMet url creation 🤦 . The last push should give the following:

library(climateR)
library(sf)
#> Warning: package 'sf' was built under R version 4.0.2
#> Linking to GEOS 3.8.1, GDAL 3.1.4, PROJ 6.3.1

# create an example point
data.frame(lon = -96.89999276, lat = 31.51999451) %>% 
  st_as_sf(coords = c("lon", "lat"), crs = 4326) %>% 
  getGridMET("tmax", startDate = "1979-01-01", endDate = "1979-01-02")
#>    source    lat       lon       date  tmax
#> 1 gridmet 31.525 -96.89167 1979-01-01 268.8
#> 2 gridmet 31.525 -96.89167 1979-01-02 272.4

# Compare to your local results!
# X28854 X28855
# 268.8  272.4

^{Created on 2021-03-09 by the reprex package (v0.3.0)}

Which matches your local results. Thank you again!

Mike

[SarahGoslee-USDA]

Thanks for your prompt fix! I can confirm that the current version matches my expectations for GridMET.