tidyverts / tsibble

Tidy Temporal Data Frames and Tools
https://tsibble.tidyverts.org
GNU General Public License v3.0
528 stars 50 forks source link

Support for intervals of day, week, month, quarter, year #242

Closed yogat3ch closed 3 years ago

yogat3ch commented 3 years ago

scan_gaps.tbl_ts passes the interval stripped of it's units due to missing handlers for intervals above "hour" in default_time_units to seq_generator. Thus scan_gaps does not work for any time periods above 1 hour. Support for day, month, week, quarter and year time intervals seems to be absent.


Brief description of the problem

d <- structure(list(time = structure(c(1546232400, 1585627200), tzone = "America/New_York", class = c("POSIXct", 
"POSIXt")), n = c(61L, 64L)), row.names = c(NA, -2L), key = structure(list(
    .rows = structure(list(1:2), ptype = integer(0), class = c("vctrs_list_of", 
    "vctrs_vctr", "list"))), row.names = c(NA, -1L), class = c("tbl_df", 
"tbl", "data.frame")), index = structure("time", ordered = TRUE), index2 = "time", interval = structure(list(
    year = 1, quarter = 0, month = 0, week = 0, day = 0, hour = 0, 
    minute = 0, second = 0, millisecond = 0, microsecond = 0, 
    nanosecond = 0, unit = 0), .regular = TRUE, class = c("interval", 
"vctrs_rcrd", "vctrs_vctr")), class = c("tbl_ts", "tbl_df", "tbl", 
"data.frame"))
tsibble::scan_gaps(d)

I would expect this to output two more rows with "2019-12-31" and "2020-12-31" as the time, but instead it allocates an enormous data.frame with ever missing minute, taking quite a long time to complete (at least on my machine).

I'm currently working on a solution proposal. Will tag the PR here when completed.

yogat3ch commented 3 years ago

Using the test data here the solution can be tested by adding the following:

test <-  purrr::map(test, as_tsibble, regular = F)
test_gaps <- purrr::map(test, scan_gaps)

If ya'll decide to integrate this functionality I'd be happy to mimic it in interval_pull such that it also behaves similarly.

earowang commented 3 years ago

Please see https://github.com/tidyverts/tsibble/issues/237