Degrees of freedom

[r-glennie]

Degrees of freedom do not make sense.

Run gam_hmm.R in inst/examples/gams.

The mod when fitted reports effective degrees of freedom of 194.36 (when you run logLik(mod)), but the maximum degrees of freedom is 9 for the spline plus 4 from the rest of the model, so 13.

Not sure, but might have been caused by most recent commit on this. @TheoMichelot

[TheoMichelot]

Thanks for pointing this out Richard, will look into it.

[r-glennie]

There are two issues with this. I have created a solution on a new branch, commit 56471dba157988d19380ce1cae37aba2a65419e1.

Takeaway points

(1) I think conditional likelihood used when computing conditional AIC should not include the penalty term from the smooths, only the data likelihood. See my inclusion of include_smooths in tmb_dat.

(2) We have misunderstood what I represents in Wood et al. (2016), e.g., in Equation (10). For regular GAMs that are linear models, I is the Hessian and so is proportional to t(X) %*% X. But in our case, the hessian is of the HMM log-likelihood. In short, we should use the joint TMB object to compute the hessian.

Hand-wavy Justification

The way I think about it: penalised log-likelihood = HMM log-likelihood + lambda * penalty.
Let V be the covariance for the spline coefficients computed from jointPrecision. Let H be the Hessian of the spline coefficients from the joint TMB object of only the HMM log-likelihood part.

When lambda is zero, V and inverse of H are the same, so V * inverse H = identity matrix and so trace(V*inverse H) = number of parameters, i.e, completely unpenalised.

As lambda -> infinity, V tends to zero and H will tend to some constant, so V * inverse H = zero matrix and so trace(V*inverse H) = 0, i.e., completely penalised.

Hence, overall, we are comparing the variance under no penalisation, inverse H, to the variance under the estimated penalty, V. This determines effective degrees of freedom.

[TheoMichelot]

Sounds great Richard, I'm happy for you to merge this into the master branch.