Automatically drop rank deficient regressors with warning

[robjhyndman]

Based on https://stackoverflow.com/q/78388045/144157

library(fable)
library(tsibble)
library(dplyr)

set.seed(123)

# create data
aa <- tibble(idx = 1:12, group = "aa", y = rnorm(12))
bb <- tibble(idx = 7:12, group = "bb", y = rnorm(6))
cc <- tibble(idx = 1:6, group = "cc", y = rnorm(6))
xreg <- tibble(idx = 1:12, x1 = c(0, 0, 1, rep(0, 9)), x2 = c(rep(0, 9), 1, 0, 0))
dat <- bind_rows(aa, bb, cc) |>
  inner_join(xreg, by = "idx") |>
  as_tsibble(index = idx, key = group)

dat |>
  model(arima = ARIMA(y ~ 1 + x1))
#> Warning: Provided exogenous regressors are rank deficient, removing regressors:
#> `x1`
#> Warning: 1 error encountered for arima
#> [1] subscript out of bounds
#> # A mable: 3 x 2
#> # Key:     group [3]
#>   group                       arima
#>   <chr>                     <model>
#> 1 aa    <LM w/ ARIMA(0,0,0) errors>
#> 2 bb                   <NULL model>
#> 3 cc    <LM w/ ARIMA(0,0,0) errors>

^{Created on 2024-05-12 with reprex v2.1.0}

Session info

``` r sessioninfo::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.4.0 (2024-04-24) #> os KDE neon 6.0 #> system x86_64, linux-gnu #> ui X11 #> language (EN) #> collate en_AU.UTF-8 #> ctype en_AU.UTF-8 #> tz Australia/Melbourne #> date 2024-05-12 #> pandoc 3.1.11.1 @ /usr/bin/ (via rmarkdown) #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date (UTC) lib source #> anytime 0.3.9 2020-08-27 [1] RSPM #> cli 3.6.2 2023-12-11 [1] RSPM #> colorspace 2.1-0 2023-01-23 [1] RSPM #> digest 0.6.35 2024-03-11 [1] RSPM #> distributional 0.4.0 2024-02-07 [1] RSPM (R 4.3.0) #> dplyr * 1.1.4 2023-11-17 [1] RSPM #> ellipsis 0.3.2 2021-04-29 [1] RSPM #> evaluate 0.23 2023-11-01 [1] RSPM #> fable * 0.3.4.9000 2024-04-28 [1] Github (tidyverts/fable@39665f2) #> fabletools * 0.4.2 2024-04-22 [1] RSPM (R 4.4.0) #> fansi 1.0.6 2023-12-08 [1] RSPM #> fastmap 1.1.1 2023-02-24 [1] RSPM #> feasts 0.3.2.9000 2024-04-28 [1] Github (tidyverts/feasts@6e9266e) #> fs 1.6.4 2024-04-25 [1] RSPM (R 4.4.0) #> generics 0.1.3 2022-07-05 [1] RSPM #> ggplot2 3.5.1 2024-04-23 [1] RSPM (R 4.3.3) #> glue 1.7.0 2024-01-09 [1] RSPM #> gtable 0.3.5 2024-04-22 [1] RSPM (R 4.3.3) #> htmltools 0.5.8.1 2024-04-04 [1] RSPM (R 4.3.3) #> knitr 1.46 2024-04-06 [1] RSPM #> lattice 0.22-6 2024-03-20 [1] RSPM #> lifecycle 1.0.4 2023-11-07 [1] RSPM #> lubridate 1.9.3 2023-09-27 [1] RSPM #> magrittr 2.0.3 2022-03-30 [1] RSPM #> munsell 0.5.1 2024-04-01 [1] RSPM #> nlme 3.1-164 2023-11-27 [1] RSPM #> pillar 1.9.0 2023-03-22 [1] RSPM #> pkgconfig 2.0.3 2019-09-22 [1] RSPM #> progressr 0.14.0 2023-08-10 [1] RSPM #> purrr 1.0.2 2023-08-10 [1] RSPM #> R.cache 0.16.0 2022-07-21 [1] RSPM (R 4.3.3) #> R.methodsS3 1.8.2 2022-06-13 [1] RSPM (R 4.3.3) #> R.oo 1.26.0 2024-01-24 [1] RSPM (R 4.3.3) #> R.utils 2.12.3 2023-11-18 [1] RSPM (R 4.3.3) #> R6 2.5.1 2021-08-19 [1] RSPM #> Rcpp 1.0.12 2024-01-09 [1] RSPM #> reprex 2.1.0 2024-01-11 [1] RSPM #> rlang 1.1.3 2024-01-10 [1] RSPM #> rmarkdown 2.26.2 2024-05-01 [1] Github (rstudio/rmarkdown@3d99b8e) #> rstudioapi 0.16.0 2024-03-24 [1] RSPM #> scales 1.3.0 2023-11-28 [1] RSPM #> sessioninfo 1.2.2 2021-12-06 [1] RSPM #> styler 1.10.3 2024-04-07 [1] RSPM (R 4.3.3) #> tibble 3.2.1 2023-03-20 [1] RSPM #> tidyr 1.3.1 2024-01-24 [1] RSPM #> tidyselect 1.2.1 2024-03-11 [1] RSPM #> timechange 0.3.0 2024-01-18 [1] RSPM #> tsibble * 1.1.4 2024-04-28 [1] Github (tidyverts/tsibble@9e0057e) #> urca 1.3-3 2022-08-29 [1] RSPM #> utf8 1.2.4 2023-10-22 [1] RSPM #> vctrs 0.6.5 2023-12-01 [1] RSPM #> withr 3.0.0 2024-01-16 [1] RSPM #> xfun 0.43 2024-03-25 [1] RSPM (R 4.3.3) #> yaml 2.3.8 2023-12-11 [1] RSPM #> #> [1] /usr/local/lib/R/site-library #> [2] /usr/lib/R/site-library #> [3] /usr/lib/R/library #> #> ────────────────────────────────────────────────────────────────────────────── ```

[mitchelloharawild]

Thanks, I think it should behave like lm() and warn that the rank deficient regressors have been removed.

[robjhyndman]

It already gives a warning. The point is that the model with all regressors removed should still be fitted, not return a NULL.

[mitchelloharawild]

Currently it errors (and returns a NULL model) if any of the regressors are rank deficient. In case there is a subset of regressors with full rank, it should use those to estimate the model with a warning.