Date ranges get chopped off by one

[geanders]

Right now, the data that we get for daily measures does not include the first day or last day of the date range, just the days within the date range. We should change to not chop off this first and last value.

[geanders]

For example, see the vignette for daily weather for Hurricane Andrew:

andrew_precip <- daily_fips(fips = "12086", date_min = "1992-08-01", date_max = "1992-08-31", 
                                             var = "PRCP")

gives

head(andrew_precip$daily_data)
#> Source: local data frame [6 x 3]
#> Groups: date [6]
#> 
#>         date     prcp prcp_reporting
#>       <date>    <dbl>          <int>
#> 1 1992-08-02 8.850000              6
#> 2 1992-08-03 9.366667              6
#> 3 1992-08-04 5.483333              6
#> 4 1992-08-05 2.716667              6
#> 5 1992-08-06 1.633333              6
#> 6 1992-08-07 7.200000              6

while we really want it to start with 1992-08-01.

[leighseverson]

I think this is originating from meteo_pull_monitors() - as far as I can tell, in ghcnd_search(), the following is the potential culprit -

if (!is.null(date_min)) {
    dat <- lapply(dat, function(z) z %>% dplyr::filter(date > date_min))
  }
  if (!is.null(date_max)) {
    dat <- lapply(dat, function(z) z %>% dplyr::filter(date < date_max))
  }

When I changed this to

 if (!is.null(date_min)) {
    dat <- lapply(dat, function(z) z %>% dplyr::filter(date >= date_min))
  }
  if (!is.null(date_max)) {
    dat <- lapply(dat, function(z) z %>% dplyr::filter(date <= date_max))
  }

I got the correct date range from meteo_pull_monitors!

 id       date  prcp
         <chr>     <date> <dbl>
1  USC00083909 1992-08-01    13
2  USC00083909 1992-08-02    48
3  USC00083909 1992-08-03    13
4  USC00083909 1992-08-04     0
5  USC00083909 1992-08-05    76
6  USC00083909 1992-08-06    10
7  USC00083909 1992-08-07     0
8  USC00083909 1992-08-08     0
9  USC00083909 1992-08-09    99
10 USC00083909 1992-08-10     0

  id       date  prcp
        <chr>     <date> <dbl>
1 USW00092811 1992-08-26     0
2 USW00092811 1992-08-27     0
3 USW00092811 1992-08-28    71
4 USW00092811 1992-08-29     0
5 USW00092811 1992-08-30     0
6 USW00092811 1992-08-31    89

[geanders]

Perfect, I added that to our pull request for rnoaa, and my branch of that geanders/rnoaa) now has that code changed.

---

From: Rachel Severson notifications@github.com Sent: Thursday, August 18, 2016 12:04:53 PM To: leighseverson/countyweather Cc: Anderson,Brooke; Author Subject: Re: [leighseverson/countyweather] Date ranges get chopped off by one (#66)

I think this is originating from meteo_pull_monitors() - as far as I can tell, in ghcnd_search(), the following is the potential culprit -

if (!is.null(date_min)) { dat <- lapply(dat, function(z) z %>% dplyr::filter(date > date_min)) } if (!is.null(date_max)) { dat <- lapply(dat, function(z) z %>% dplyr::filter(date < date_max)) }

When I changed this to

if (!is.null(date_min)) { dat <- lapply(dat, function(z) z %>% dplyr::filter(date >= date_min)) } if (!is.null(date_max)) { dat <- lapply(dat, function(z) z %>% dplyr::filter(date <= date_max)) }

I got the correct date range from meteo_pull_monitors!

id date prcp

1 USC00083909 1992-08-01 13 2 USC00083909 1992-08-02 48 3 USC00083909 1992-08-03 13 4 USC00083909 1992-08-04 0 5 USC00083909 1992-08-05 76 6 USC00083909 1992-08-06 10 7 USC00083909 1992-08-07 0 8 USC00083909 1992-08-08 0 9 USC00083909 1992-08-09 99 10 USC00083909 1992-08-10 0 id date prcp 1 USW00092811 1992-08-26 0 2 USW00092811 1992-08-27 0 3 USW00092811 1992-08-28 71 4 USW00092811 1992-08-29 0 5 USW00092811 1992-08-30 0 6 USW00092811 1992-08-31 89 ## You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/leighseverson/countyweather/issues/66#issuecomment-240806261, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFVFiNujivxM3bTRnkiZP4aqJojHTtxPks5qhJ7FgaJpZM4JnqZH.