Adjusted series

[gumlese]

First of all, thank you for an amazing package.

The issue I am having is that the final series produced by regARIMA does not have outlier adjustments applied. This is affecting outputs, like trend-cycle estimation and long-term trend extraction, later in the process.

Take the following example using the electricity equipment time series from the fpp package. There is a level shift identified in January 2009 that should be shifting the final series down. I can extract an adjusted trend-cycle from the X13 html output, but there is no associated long term trend.

library(fpp) m <- seas(elecequip, outlier.types = "LS", seats.hpcycle = "YES", seats.save = "ltt") plot(series(m,"a19")) lines(final(m)) lines(series(m,"s12")) lines(series(m,"ltt"))

Cheers

Sean

[christophsax]

Sorry for the belated answer.

Not sure if I got the problem, but outlier are removed before modelling the ARIMA model and are reapplied later on, so they are part of the final series.

This is why the final series has the same level as the original one both after and before the level shift.

library(seasonal)
library(fpp)
#> Loading required package: forecast
#> Loading required package: fma
#> Loading required package: expsmooth
#> Loading required package: lmtest
#> Loading required package: zoo
#> 
#> Attaching package: 'zoo'
#> The following objects are masked from 'package:base':
#> 
#>     as.Date, as.Date.numeric
#> Loading required package: tseries
m <- seas(elecequip, outlier.types = "LS", seats.hpcycle = "YES", seats.save = "ltt")
plot(m)

[gumlese]

Hey Christoph,

Thanks for the response. Sorry, I didn't present my issue clearly!

I see how the outliers are removed and reintroduced for the final set of series from SEATS after being identified from the ARIMA model. However, shouldn’t the outliers also be removed from the trend-cycle series feeding into the HP filter to extract the long-term trend (and subsequently re-introduced)?

Sean

[christophsax]

I haven't worked with the SEATS HP decomposition so far, and just did some quick experiments, so take the following with a grain of salt.

The way I see it is that outliers are removed during for the SEATS decomposition, and added to the final trend decomposition. If you do the HP cycle/longterm decomposition, the outliers are added to the cycle. So the actual HP decomposition looks like this:

library(tsbox)  # devtools::install_github("christophsax/tsbox")
suppressMessages(library(fpp))
library(seasonal) 
m <- seas(elecequip, outlier.types = "LS", seats.hpcycle = "yes", seats.save = "ltt")

ts_plot(
  `Adjusted Overall Trend` = series(m, "seats.trend") / series(m, "regression.outlier"),
  `Adjusted Cycle` = series(m, "seats.cycle") / series(m, "regression.outlier"),
  `Long Term Trend` = series(m, "seats.longtermtrend")
)

Outliers are then added to to the cycle and the overall trend, to produce the final series. Does that make sense?

[christophsax]

Sorry, seems not to be true what I said: The HP decomposition is really just the decompostion of the seats.trend series (which includes the outlier) into a longterm and a cycle component, nothing else. Here is an (almost) reproduction with the HP filter from the mFilter package:

library(tsbox)  # devtools::install_github("christophsax/tsbox")
suppressMessages(library(fpp))
library(seasonal) 
library(mFilter)
m <- seas(elecequip, outlier.types = "LS", seats.hpcycle = "yes", seats.save = "ltt")

ts_plot(
  `Trend mfilter` = mFilter(series(m, "seats.trend"), filter="HP")$trend,
  `Trend X13` = series(m, "seats.longtermtrend"),
  `Cycle mfilter` = mFilter(series(m, "seats.trend"), filter="HP")$cycle + 100,
  `Cycle X13` = series(m, "seats.cycle")
)

[gumlese]

Hey Christoph, this issue can be closed because it is a matter for the US Census program. I talked to the US Census a while back and they agreed the trend-cycle decomposition should not have outliers included. Maybe this will flow through to the seasonal package one day but mfilter is sufficient.

Thanks for your help.