Implement the AutoARIMA algorithm

[azev77]

@guilhermebodin great idea! Please benchmark it w/ ARCHModels.jl

using ARCHModels;
auto_arma(df, ic) = selectmodel(ARCH{0}, df, meanspec=ARMA, criterion=ic, minlags=0, maxlags=10);

[guilhermebodin]

Hi @azev77 I am not so sure about how to do a proper benchmark

using StateSpaceModels, ARCHModels
serie = randn(1000)
@time am = selectmodel(ARCH{0}, serie;  meanspec=ARMA);
loglikelihood(am)
aic(am)
@time model = auto_arima(serie)
loglike(model)
model.results.aic

In this simple example curiously ARCHModels.jl tends to choose p and q greater than 0 every time. ARCHModels.jl is certainly faster, probably they use a different estimation method.

After testing in BG96, StateSpaceModels.NILE and StateSpaceModels.INTERNET I came to the conclusion that even with very similar models they usually converge to different log-likelihoods.

Here I show an example

using StateSpaceModels, ARCHModels, CSV, DataFrames
internet = CSV.File(StateSpaceModels.INTERNET) |> DataFrame
serie = internet.dinternet[2:end]
@time am = fit(ARCH{0}, serie, meanspec=ARMA{1,1})
loglikelihood(am)
@time begin
    model = SARIMA(serie; order = (1,0,1), include_mean = true)
    fix_hyperparameters!(model, Dict("ar_L1" => 0.631033, "sigma2_η" => 10.1978, "ma_L1" => 0.501191, "mean" => 0.60756))
    StateSpaceModels.handle_optim_initial_hyperparameters(model)
    loglike(model)
end

[azev77]

@s-broda maybe knows why

[s-broda]

Venturing a guess, it might be due to different handling of presample values? ARCHModels just uses the sample variance and sample mean to start the recursion.

[guilhermebodin]

@s-broda does ARCHModels.jl use a Kalman Filter to estimate the hyperparameters?

[s-broda]

No.

[guilhermebodin]

Probably that is why it is faster.

[azev77]

Btw, I strongly believe proper hyper-tuning should be based on out of sample fit

[guilhermebodin]

This is an interesting discussion. In StateSpaceModels.jl we have a benchmark function that evaluates the skill of probabilistic forecasts generated by the models.

[guilhermebodin]

I am closing the issue but we can continue the discussion here.