Add support for reconciling disjoint hierarchies

[robjhyndman]

This seems to be unstable for some reason (non-positive-definite covariance), perhaps the .id is being used in reconciliation in some way.

library(fpp3)
tourism_stretch <- tourism %>%
  stretch_tsibble(.init = 16)
fits <- tourism_stretch %>%
  model(ets = ETS(Trips)) %>%
  reconcile(ets_adj = min_trace(ets, method='wls'))
fc <- fits %>% forecast(h = 1)
fc %>% accuracy(tourism) %>%
  group_by(.model) %>%
  summarise(MASE = mean(MASE))

[mitchelloharawild]

From an email I sent related to this issue:

A simple test for reconciliation on a stretched tsibble would be as follows:

• stretch the tsibble (2 folds should be sufficient), then make reconciled forecasts
• Filter the tsibble to obtain a tsibble for each fold above, then separately produce reconciled forecasts using the same method above.

The resulting forecasts should be identical, if not the fold ID is probably messing with the reconciliation (actually, after thinking about it in the paragraph below, I know that there is a problem with this now!).

Thinking about this has actually led me to an interesting thought about reconciliation of disjoint graph structures. What we actually have is separate network structures for each fold, and so the reconciliation should probably happen independently for each network. This has neat conceptual implications with reconciling a set of models without any specified aggregation structure - all bottom level time series are separate networks and so there is nothing to reconcile. I'll have to think about this some more, but if we can detect disjoint graph structures within the key variable (should be possible) and then reconcile each tree separately (with their own S matrix, etc.), we should be able to keep the same interface.

[mitchelloharawild]

An update. It is now possible to create these disjoint sets using aggregate_key. As suggested by Rob, this uses the same hierarchical interface as lm(). Below I've nested key by .id, and removed the top level by excluding the intercept. I've also made the reconciliation fail when it detects disjoint sets, until the reconciliation of multiple hierarchies is implemented.

library(fable)
#> Loading required package: fabletools
library(tsibble)
lung_deaths_long <- as_tsibble(cbind(mdeaths, fdeaths))

lung_deaths_long %>% 
  aggregate_key(key, value = sum(value)) %>% 
  stretch_tsibble(.init = 71)
#> # A tsibble: 429 x 4 [1M]
#> # Key:       .id, key [6]
#>    key        index value   .id
#>    <chr>      <mth> <dbl> <int>
#>  1 fdeaths 1974 Jan   901     1
#>  2 fdeaths 1974 Feb   689     1
#>  3 fdeaths 1974 Mar   827     1
#>  4 fdeaths 1974 Apr   677     1
#>  5 fdeaths 1974 May   522     1
#>  6 fdeaths 1974 Jun   406     1
#>  7 fdeaths 1974 Jul   441     1
#>  8 fdeaths 1974 Aug   393     1
#>  9 fdeaths 1974 Sep   387     1
#> 10 fdeaths 1974 Oct   582     1
#> # … with 419 more rows

lung_deaths_long %>% 
  stretch_tsibble(.init = 71) %>% 
  aggregate_key(0 + .id/key, value = sum(value))
#> # A tsibble: 429 x 4 [1M]
#> # Key:       .id, key [6]
#>    .id   key             index value
#>    <int> <chr>           <mth> <dbl>
#>  1 1     <aggregated> 1974 Jan  3035
#>  2 2     <aggregated> 1974 Jan  3035
#>  3 1     <aggregated> 1974 Feb  2552
#>  4 2     <aggregated> 1974 Feb  2552
#>  5 1     <aggregated> 1974 Mar  2704
#>  6 2     <aggregated> 1974 Mar  2704
#>  7 1     <aggregated> 1974 Apr  2554
#>  8 2     <aggregated> 1974 Apr  2554
#>  9 1     <aggregated> 1974 May  2014
#> 10 2     <aggregated> 1974 May  2014
#> # … with 419 more rows

lung_deaths_long %>% 
  aggregate_key(key, value = sum(value)) %>% 
  stretch_tsibble(.init = 71) %>% 
  model(mdl = ETS(value)) %>% 
  reconcile(mdl = min_trace(mdl)) %>% 
  forecast()
#> Warning: Reconciliation in fable is highly experimental. The interface will
#> likely be refined in the near future.
#> Loading required namespace: SparseM
#> Reconciliation of disjoint hierarchical structures is not yet supported.

^{Created on 2019-08-13 by the reprex package (v0.3.0)}