search

tidyverts / fabletools

General fable features useful for extension packages
http://fabletools.tidyverts.org/
89 stars 31 forks source link

accuracy function cannot validate fable #215

Closed ssoftcheck closed 4 years ago

ssoftcheck commented 4 years ago

I have a mable for a variable called "target"

models = model_data %>% as_tsibble(index=date) %>% filter(date < train.limit) %>%
      model(
            arima=ARIMA(target ~ pdq(p=0:5, d=0:1, q=0:5) + PDQ(P=0:1,D=0:1,Q=0:1), ic = "aicc", greedy = FALSE, stepwise = FALSE)
            ,arima_log=ARIMA(log(target) ~ pdq(p=0:5, d=0:2, q=0:5) + PDQ(P=0:1, D=0:1, Q=0:1), ic="aicc", greedy=FALSE, stepwise=FALSE)
            ,arima_trend=ARIMA(target ~ trend() + pdq(p=0:5, d=0:1, q=0:5) + PDQ(P=0:1,D=0:1,Q=0:1), ic = "aicc", greedy = FALSE, stepwise = FALSE)
            ,arima_log_trend=ARIMA(log(target) ~ trend() + pdq(p=0:5, d=0:2, q=0:5) + PDQ(P=0:1, D=0:1, Q=0:1), ic="aicc", greedy=FALSE, stepwise=FALSE)
            )

When I try to compute accuracy on the full data

models %>% forecast(h=train.limit.days) %>% accuracy(model_data %>% as_tsibble(index=date))

I get the error "Could not find response variable(s) in the fable: target", although the variable "target" exists. I stepped through and when the validation occurs the target column exists as ".fc" and the distribution column exists as ".dist"

models %>% forecast(h=train.limit.days)
# A fable: 28 x 4 [1D]
# Key:     .model [4]
   .model    date         target .distribution      
   <chr>     <date>        <dbl> <dist>             
 1 arima     2020-06-02 1830175. N(1830175, 3.5e+06)
 2 arima     2020-06-03 1849998. N(1849998, 1.5e+07)
 3 arima     2020-06-04 1871458. N(1871458, 4.0e+07)
 4 arima     2020-06-05 1894747. N(1894747, 8.3e+07)
 5 arima     2020-06-06 1916946. N(1916946, 1.5e+08)
 6 arima     2020-06-07 1937011. N(1937011, 2.4e+08)
 7 arima     2020-06-08 1956764. N(1956764, 3.6e+08)
 8 arima_log 2020-06-02 1850969. t(N(14, 0.016))    
 9 arima_log 2020-06-03 1888530. t(N(14, 0.032))    
10 arima_log 2020-06-04 1928561. t(N(14, 0.052))

rlang trace

<error/rlang_error>
Could not find response variable(s) in the fable: target
Backtrace:
     █
  1. └─`%>%`(...)
  2.   ├─base::withVisible(eval(quote(`_fseq`(`_lhs`)), env, env))
  3.   └─base::eval(quote(`_fseq`(`_lhs`)), env, env)
  4.     └─base::eval(quote(`_fseq`(`_lhs`)), env, env)
  5.       └─`_fseq`(`_lhs`)
  6.         └─magrittr::freduce(value, `_function_list`)
  7.           ├─base::withVisible(function_list[[k]](value))
  8.           └─function_list[[k]](value)
  9.             ├─fabletools::accuracy(., model_data %>% as_tsibble(index = date))
 10.             └─fabletools:::accuracy.fbl_ts(., model_data %>% as_tsibble(index = date))
 11.               ├─dplyr::transmute(object, .fc = !!resp, .dist = !!dist, !!!syms(by))
 12.               ├─tsibble:::transmute.tbl_ts(...)
 13.               │ └─tsibble:::bind_tsibble(NextMethod(), .data, position = "before")
 14.               │   └─tibble::as_tibble(data)
 15.               ├─base::NextMethod()
 16.               └─dplyr:::transmute.data.frame(...)
 17.                 ├─dplyr::mutate(.data, ..., .keep = "none")
 18.                 └─fabletools:::mutate.fbl_ts(.data, ..., .keep = "none")
 19.                   ├─fabletools::as_fable(...)
 20.                   └─fabletools:::as_fable.tbl_ts(...)
 21.                     └─fabletools:::validate_fable(fbl)
mitchelloharawild commented 4 years ago

If you've updated to the latest version(s) of tsibble (v0.9.0) or dplyr (v1.0.0), the current CRAN version of fabletools is incompatible with them.

The development version has fixed these issues, and I am working on getting the update on CRAN.

ilebizard commented 4 years ago

Hi Mitchel,

I just tried the development versions of tsibble (v0.9.0.900), fabletools (0.2.0), fable (0.2.1). There is still an issue when using accuracy(validation_set) with a message below: Error in vec_c(...) : Error in vec_c(...) : Internal error in vec_proxy_assign_opts(): proxy of type double incompatible with value proxy of type integer.

Meanwhile, accuracy() applied to the training set works fine.

It is also noted here that forecast() works only when fitting one model, e.g. fitModel = train_data %>% model (arima = ARIMA(y)). If there are more than 1 models, e.g. fitModel = train_data %>% model (trend = TSLM(y ~ trend()), arima = ARIMA(y)), there is an error message:

Error in vctrs::vec_rbind(!!!lst_col) : Error in vctrs::vec_rbind(!!!lst_col) : Internal error in vec_proxy_assign_opts(): proxy of type double incompatible with value proxy of type integer.

Thank you! Nguyen

mitchelloharawild commented 4 years ago

Could you post a reproducible example that includes your session info? This can be done with reprex::reprex(si = TRUE).

ilebizard commented 4 years ago

Sorry, my project is entirely on Databricks with big datasets read from a database, so not sure how to post it here.

mitchelloharawild commented 4 years ago

Sure, could you instead post the results of sessionInfo()?

ilebizard commented 4 years ago

Please find results of sessionInfo() below:

R version 3.6.2 (2019-12-12) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 16.04.6 LTS

Matrix products: default BLAS: /usr/lib/openblas-base/libblas.so.3 LAPACK: /usr/lib/libopenblasp-r0.2.18.so

locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] data.table_1.12.8 fable_0.2.1 feasts_0.1.4 fabletools_0.2.0
[5] tsibbledata_0.2.0 tsibble_0.9.0.9000 lubridate_1.7.4 fpp3_0.3
[9] forcats_0.4.0 stringr_1.4.0 dplyr_1.0.0 purrr_0.3.3
[13] readr_1.3.1 tidyr_1.0.0 tibble_3.0.1 ggplot2_3.2.1
[17] tidyverse_1.3.0 sparklyr_1.0.4 SparkR_2.4.5

loaded via a namespace (and not attached): [1] Rcpp_1.0.2 lattice_0.20-38 forge_0.2.0 assertthat_0.2.1
[5] rprojroot_1.3-2 digest_0.6.25 utf8_1.1.4 R6_2.4.0
[9] cellranger_1.1.0 backports_1.1.5 reprex_0.3.0 httr_1.4.1
[13] pillar_1.4.4 TeachingDemos_2.10 rlang_0.4.6 lazyeval_0.2.2
[17] readxl_1.3.1 rstudioapi_0.11 r2d3_0.2.3 htmlwidgets_1.5.1 [21] munsell_0.5.0 broom_0.5.6 anytime_0.3.7 compiler_3.6.2
[25] modelr_0.1.8 pkgconfig_2.0.3 base64enc_0.1-3 Rserve_1.8-6
[29] urca_1.3-0 htmltools_0.4.0 tidyselect_1.1.0 fansi_0.4.0
[33] crayon_1.3.4 dbplyr_1.4.2 withr_2.1.2 grid_3.6.2
[37] nlme_3.1-144 jsonlite_1.6.1 gtable_0.3.0 lifecycle_0.2.0
[41] DBI_1.0.0 magrittr_1.5 scales_1.1.1 cli_2.0.2
[45] stringi_1.4.3 hwriter_1.3.2 fs_1.3.1 xml2_1.2.2
[49] ellipsis_0.3.0 generics_0.0.2 vctrs_0.3.1 hwriterPlus_1.0-3 [53] tools_3.6.2 glue_1.4.1 hms_0.5.3 colorspace_1.4-1
[57] rvest_0.3.5 haven_2.3.1

mitchelloharawild commented 4 years ago

While it looks like you're using fabletools v0.2.0, the output suggests that you are using v0.1.3. Have you recently updated your packages? If so, perhaps the fable was created using an older version of fabletools, and you are trying to compute the accuracy of it using the newer version.

ilebizard commented 4 years ago

Thanks, Mitchell. To make sure, I reinstalled tsibble, fabletools, fable, and feasts in this order, and reran. There is still an error message of " Error in vctrs::vec_rbind(!!!lst_col) : Error in vctrs::vec_rbind(!!!lst_col) : Internal error in vec_proxy_assign_opts(): proxy of type double incompatible with value proxy of type integer."

Below is what I loaded and what sessionInfo said: library(SparkR) library(sparklyr) library(tidyverse)

library(fpp3)

library(forecast)

library(tsibble) library(fabletools) library(fable) library(feasts)

library(vctrs)

library(tseries)

library(schoolmath)

library(data.table)

sessionInfo()

R version 3.6.2 (2019-12-12) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 16.04.6 LTS

Matrix products: default BLAS: /usr/lib/openblas-base/libblas.so.3 LAPACK: /usr/lib/libopenblasp-r0.2.18.so

locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] data.table_1.12.8 feasts_0.1.4 fable_0.2.1 fabletools_0.2.0
[5] tsibble_0.9.0.9000 forcats_0.4.0 stringr_1.4.0 dplyr_1.0.0
[9] purrr_0.3.4 readr_1.3.1 tidyr_1.1.0 tibble_3.0.1
[13] ggplot2_3.3.1 tidyverse_1.3.0 sparklyr_1.0.4 SparkR_2.4.5
[17] remotes_2.1.1 devtools_2.3.0 usethis_1.6.1

loaded via a namespace (and not attached): [1] nlme_3.1-144 fs_1.3.1 lubridate_1.7.9
[4] httr_1.4.1 rprojroot_1.3-2 tools_3.6.2
[7] backports_1.1.7 R6_2.4.1 DBI_1.0.0
[10] colorspace_1.4-1 withr_2.2.0 tidyselect_1.1.0
[13] prettyunits_1.1.1 processx_3.4.2 curl_4.2
[16] compiler_3.6.2 progressr_0.6.0 cli_2.0.2
[19] rvest_0.3.5 forge_0.2.0 xml2_1.2.2
[22] desc_1.2.0 scales_1.1.1 callr_3.4.3
[25] digest_0.6.25 base64enc_0.1-3 pkgconfig_2.0.3
[28] htmltools_0.4.0 sessioninfo_1.1.1 dbplyr_1.4.2
[31] htmlwidgets_1.5.1 rlang_0.4.6 readxl_1.3.1
[34] rstudioapi_0.11 generics_0.0.2 farver_2.0.3
[37] hwriter_1.3.2 jsonlite_1.6.1 distributional_0.1.0 [40] magrittr_1.5 Rcpp_1.0.4.6 munsell_0.5.0
[43] fansi_0.4.1 lifecycle_0.2.0 stringi_1.4.6
[46] pkgbuild_1.0.8 grid_3.6.2 crayon_1.3.4
[49] lattice_0.20-38 haven_2.3.1 hms_0.5.3
[52] anytime_0.3.7 ps_1.3.3 pillar_1.4.4
[55] Rserve_1.8-6 pkgload_1.1.0 reprex_0.3.0
[58] urca_1.3-0 glue_1.4.1 modelr_0.1.8
[61] vctrs_0.3.1 testthat_2.3.2 cellranger_1.1.0
[64] gtable_0.3.0 assertthat_0.2.1 TeachingDemos_2.10
[67] r2d3_0.2.3 broom_0.5.6 memoise_1.1.0
[70] ellipsis_0.3.1 hwriterPlus_1.0-3

ilebizard commented 4 years ago

Note that the error message: "Error in vctrs::vec_rbind(!!!lst_col) : Error in vctrs::vec_rbind(!!!lst_col) : Internal error in vec_proxy_assign_opts(): proxy of type double incompatible with value proxy of type integer." popped up after I called fitModel %>% forecast(h) when I tried to fit more than 1 model in fitModel = model(model1 =….., model2= ……).

If fitting just one model, then forecast(h) worked, but then accuracy(test_set) did not: " Error in vec_c(...) : Error in vec_c(...) : Internal error in vec_proxy_assign_opts(): proxy of type double incompatible with value proxy of type integer."

Thank you!

mitchelloharawild commented 4 years ago

Let's try to simplify the problem by using a smaller dataset we both have access to. Can you see if this code works for you?

library(fable)
#> Loading required package: fabletools
library(tsibble)
library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union
model_data <- as_tsibble(cbind(mdeaths, fdeaths)) %>% rename(date = index, target = value)

train.limit <- yearmonth("1979 Jan")
models = model_data %>% filter(date < train.limit) %>%
  model(
    arima=ARIMA(target ~ pdq(p=0:5, d=0:1, q=0:5) + PDQ(P=0:1,D=0:1,Q=0:1), ic = "aicc", greedy = FALSE, stepwise = FALSE)
    ,arima_log=ARIMA(log(target) ~ pdq(p=0:5, d=0:2, q=0:5) + PDQ(P=0:1, D=0:1, Q=0:1), ic="aicc", greedy=FALSE, stepwise=FALSE)
    ,arima_trend=ARIMA(target ~ trend() + pdq(p=0:5, d=0:1, q=0:5) + PDQ(P=0:1,D=0:1,Q=0:1), ic = "aicc", greedy = FALSE, stepwise = FALSE)
    ,arima_log_trend=ARIMA(log(target) ~ trend() + pdq(p=0:5, d=0:2, q=0:5) + PDQ(P=0:1, D=0:1, Q=0:1), ic="aicc", greedy=FALSE, stepwise=FALSE)
  )

models %>% forecast(h=12) %>% accuracy(model_data)
#> # A tibble: 8 x 10
#>   .model          key     .type     ME  RMSE   MAE    MPE  MAPE  MASE    ACF1
#>   <chr>           <chr>   <chr>  <dbl> <dbl> <dbl>  <dbl> <dbl> <dbl>   <dbl>
#> 1 arima           fdeaths Test    3.37  36.8  27.8  0.674  5.27 0.370 -0.398 
#> 2 arima           mdeaths Test   29.6  118.   83.3  2.09   6.12 0.472 -0.424 
#> 3 arima_log       fdeaths Test  -20.3   43.3  27.7 -3.37   5.10 0.369 -0.171 
#> 4 arima_log       mdeaths Test   13.9  125.   83.4  0.586  5.77 0.473 -0.0990
#> 5 arima_log_trend fdeaths Test   12.1   46.4  33.3  0.981  5.45 0.444  0.620 
#> 6 arima_log_trend mdeaths Test  -23.1  138.  102.  -2.62   6.89 0.580 -0.0729
#> 7 arima_trend     fdeaths Test   13.3   56.8  41.5  0.686  6.95 0.553  0.669 
#> 8 arima_trend     mdeaths Test  -23.1  148.  115.  -3.03   8.09 0.649  0.162

Created on 2020-06-12 by the reprex package (v0.3.0)

Session info ``` r devtools::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 3.6.2 (2019-12-12) #> os Ubuntu 18.04.4 LTS #> system x86_64, linux-gnu #> ui X11 #> language en_AU:en #> collate en_AU.UTF-8 #> ctype en_AU.UTF-8 #> tz Australia/Melbourne #> date 2020-06-12 #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date lib source #> anytime 0.3.7 2020-01-20 [1] CRAN (R 3.6.1) #> assertthat 0.2.1 2019-03-21 [1] CRAN (R 3.6.1) #> backports 1.1.7 2020-05-13 [1] RSPM (R 3.6.3) #> callr 3.4.3 2020-03-28 [1] CRAN (R 3.6.2) #> cli 2.0.2 2020-02-28 [1] RSPM (R 3.6.2) #> colorspace 1.4-1 2019-03-18 [1] CRAN (R 3.6.1) #> crayon 1.3.4 2017-09-16 [1] CRAN (R 3.6.1) #> desc 1.2.0 2018-05-01 [1] CRAN (R 3.6.1) #> devtools 2.2.2 2020-02-17 [1] RSPM (R 3.6.2) #> digest 0.6.25 2020-02-23 [1] RSPM (R 3.6.2) #> distributional 0.1.0.9000 2020-06-10 [1] local #> dplyr * 1.0.0 2020-05-29 [1] CRAN (R 3.6.2) #> ellipsis 0.3.1 2020-05-15 [1] CRAN (R 3.6.2) #> evaluate 0.14 2019-05-28 [1] CRAN (R 3.6.1) #> fable * 0.2.1 2020-06-11 [1] local #> fabletools * 0.2.0 2020-06-11 [1] local #> fansi 0.4.1 2020-01-08 [1] RSPM (R 3.6.2) #> farver 2.0.3 2020-01-16 [1] CRAN (R 3.6.1) #> feasts 0.1.4 2020-06-04 [1] local #> fs 1.4.1 2020-04-04 [1] RSPM (R 3.6.3) #> generics 0.0.2 2018-11-29 [1] CRAN (R 3.6.1) #> ggplot2 3.3.1 2020-05-28 [1] CRAN (R 3.6.2) #> glue 1.4.1.9000 2020-05-26 [1] Github (tidyverse/glue@a605000) #> gtable 0.3.0 2019-03-25 [1] CRAN (R 3.6.1) #> highr 0.8 2019-03-20 [1] CRAN (R 3.6.1) #> htmltools 0.4.0 2019-10-04 [1] CRAN (R 3.6.1) #> knitr 1.28 2020-02-06 [1] RSPM (R 3.6.2) #> lattice 0.20-38 2018-11-04 [2] CRAN (R 3.6.2) #> lifecycle 0.2.0.9000 2020-03-19 [1] Github (r-lib/lifecycle@355dcba) #> lubridate 1.7.8 2020-04-06 [1] RSPM (R 3.6.3) #> magrittr 1.5 2014-11-22 [1] CRAN (R 3.6.1) #> memoise 1.1.0 2017-04-21 [1] CRAN (R 3.6.1) #> munsell 0.5.0 2018-06-12 [1] CRAN (R 3.6.1) #> nlme 3.1-142 2019-11-07 [2] CRAN (R 3.6.2) #> numDeriv 2016.8-1.1 2019-06-06 [1] CRAN (R 3.6.1) #> pillar 1.4.4 2020-05-25 [1] Github (r-lib/pillar@2f5ad11) #> pkgbuild 1.0.8 2020-05-07 [1] RSPM (R 3.6.3) #> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 3.6.1) #> pkgload 1.1.0 2020-05-29 [1] CRAN (R 3.6.2) #> prettyunits 1.1.1 2020-01-24 [1] CRAN (R 3.6.2) #> processx 3.4.2 2020-02-09 [1] RSPM (R 3.6.2) #> progressr 0.6.0 2020-05-19 [1] CRAN (R 3.6.2) #> ps 1.3.3 2020-05-08 [1] RSPM (R 3.6.3) #> purrr 0.3.4 2020-04-17 [1] RSPM (R 3.6.3) #> R6 2.4.1 2019-11-12 [1] CRAN (R 3.6.1) #> Rcpp 1.0.4.6 2020-04-09 [1] CRAN (R 3.6.2) #> remotes 2.1.1 2020-02-15 [1] RSPM (R 3.6.2) #> rlang 0.4.6.9000 2020-05-20 [1] Github (r-lib/rlang@691b5a8) #> rmarkdown 2.1 2020-01-20 [1] CRAN (R 3.6.2) #> rprojroot 1.3-2 2018-01-03 [1] CRAN (R 3.6.1) #> scales 1.1.1 2020-05-11 [1] RSPM (R 3.6.3) #> sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 3.6.1) #> stringi 1.4.6 2020-02-17 [1] CRAN (R 3.6.2) #> stringr 1.4.0 2019-02-10 [1] CRAN (R 3.6.1) #> testthat 2.3.2 2020-03-02 [1] RSPM (R 3.6.3) #> tibble 3.0.1 2020-04-20 [1] RSPM (R 3.6.3) #> tidyr 1.1.0 2020-05-20 [1] RSPM (R 3.6.3) #> tidyselect 1.1.0 2020-05-11 [1] RSPM (R 3.6.3) #> tsibble * 0.9.0 2020-06-02 [1] Github (tidyverts/tsibble@c837e83) #> urca 1.3-0 2016-09-06 [1] CRAN (R 3.6.1) #> usethis 1.5.1.9000 2020-01-31 [1] Github (r-lib/usethis@7d8b066) #> utf8 1.1.4 2018-05-24 [1] CRAN (R 3.6.1) #> vctrs 0.3.0.9000 2020-05-28 [1] Github (r-lib/vctrs@373e1ce) #> withr 2.2.0 2020-04-20 [1] RSPM (R 3.6.3) #> xfun 0.13 2020-04-13 [1] RSPM (R 3.6.3) #> yaml 2.2.1 2020-02-01 [1] RSPM (R 3.6.2) #> #> [1] /home/mitchell/R/x86_64-pc-linux-gnu-library/3.6 #> [2] /opt/R/3.6.2/lib/R/library ```
ssoftcheck commented 4 years ago

I just tried commit 552c764 and the accuracy issue is solved. there were some new, unexpected (to me) renaming's.

i have no issue with forecast from a mable with multiple models

@mitchelloharawild since this issue was about the accuracy() function, should I close this?

ilebizard commented 4 years ago

Thanks, but it did not work:

"Error : Could not find response variable(s) in the fable: target"

sessionInfo()

R version 3.6.2 (2019-12-12) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 16.04.6 LTS

Matrix products: default BLAS: /usr/lib/openblas-base/libblas.so.3 LAPACK: /usr/lib/libopenblasp-r0.2.18.so

locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] remotes_2.1.1 devtools_2.3.0 usethis_1.6.1 dplyr_1.0.0
[5] tsibble_0.9.0.9000 fable_0.2.1 fabletools_0.2.0

loaded via a namespace (and not attached): [1] Rcpp_1.0.4.6 urca_1.3-0 pillar_1.4.4 compiler_3.6.2
[5] TeachingDemos_2.10 prettyunits_1.1.1 tools_3.6.2 testthat_2.3.2
[9] pkgload_1.1.0 pkgbuild_1.0.8 digest_0.6.25 lubridate_1.7.9
[13] memoise_1.1.0 lifecycle_0.2.0 tibble_3.0.1 gtable_0.3.0
[17] nlme_3.1-144 anytime_0.3.7 lattice_0.20-38 pkgconfig_2.0.3
[21] rlang_0.4.6 cli_2.0.2 curl_4.2 hwriter_1.3.2
[25] withr_2.2.0 desc_1.2.0 generics_0.0.2 vctrs_0.3.1
[29] fs_1.3.1 feasts_0.1.4 rprojroot_1.3-2 grid_3.6.2
[33] tidyselect_1.1.0 glue_1.4.1 R6_2.4.1 processx_3.4.2
[37] fansi_0.4.1 sessioninfo_1.1.1 callr_3.4.3 ggplot2_3.3.1
[41] purrr_0.3.4 tidyr_1.1.0 SparkR_2.4.5 magrittr_1.5
[45] htmltools_0.4.0 hwriterPlus_1.0-3 backports_1.1.7 ps_1.3.3
[49] scales_1.1.1 ellipsis_0.3.1 assertthat_0.2.1 Rserve_1.8-6
[53] colorspace_1.4-1 munsell_0.5.0 crayon_1.3.4

mitchelloharawild commented 4 years ago

I just tried commit 552c764 and the accuracy issue is solved. there were some new, unexpected (to me) renaming's.

* ".distribution" has become the target variable name

* ".mean" has become the forecast

* unnest_tsibble() no longer works on the confidence interval, but unpack_hilo() does

i have no issue with forecast from a mable with multiple models

@mitchelloharawild since this issue was about the accuracy() function, should I close this?

Please refer to the changes documented in the news: https://fabletools.tidyverts.org/dev/news/index.html The fable has changed to emphasise distributions, which are now provided by the distributional package (http://pkg.mitchelloharawild.com/distributional/). This package makes it easy to compute point forecasts and other statistics from distributions, and so the point forecast column is now optional in the fable.

I'll make a note of unnest_tsibble() no longer working, as the data structure for hilo objects has changed (you can now access lower, upper, and level using $, or via unpack_hilo() as you suggest).

We can keep this issue open until we solve @ilebizard's problem.

mitchelloharawild commented 4 years ago

@ilebizard

Thanks, but it did not work:

"Error : Could not find response variable(s) in the fable: target"

I don't see how this is possible as this error no longer exists in the code for fabletools v0.2.0. e7510882ad40f28f9fcac12a062ed87514c46214

Can you try updating your packages again and running the minimal example above in a new R session?

ilebizard commented 4 years ago

Thanks, Mitchell. I reinstalled fresh tsibble, fabletools, and fable packages in this order and this example worked.

Quick question (just in case): If I install fable before fabletools (both are development versions), will fable pick up older fabletools that is on CRAN?

mitchelloharawild commented 4 years ago

Installing dev fable will not install older versions of package dependencies.

mitchelloharawild commented 4 years ago

Closing for lack of MRE. This appears to be an issue with using older package versions.