stan-dev / projpred

Projection predictive variable selection
https://mc-stan.org/projpred/
Other
110 stars 25 forks source link

Convergence checks for additive submodels #479

Open fweber144 opened 10 months ago

fweber144 commented 10 months ago

As mentioned in #478, the convergence checks for additive models are probably still incomplete, even with PR #478 being merged now. I have added corresponding TODO comments in the code, lines https://github.com/stan-dev/projpred/blob/97c5bea9cafd041c8b2de4dd1b3aa56a97e3e469/R/divergence_minimizers.R#L1022-L1023 and https://github.com/stan-dev/projpred/blob/97c5bea9cafd041c8b2de4dd1b3aa56a97e3e469/R/divergence_minimizers.R#L1025-L1028

@AlejandroCatalina, do you know if (and if yes, how) we could improve our check for convergence of the submodel fits from fit_gam_callback() and fit_gamm_callback()?

AlejandroCatalina commented 10 months ago

Aside from the diagnostics fit_s$gam would provide I'm not sure how much we could inject it. I would suspect we can indeed run some diagnostics on top. For GAM I'd think that similar diagnostics to GLM should work, as it's fitting an augmented GLM underneath, and for GAMM one would need to check the quality of the laplace approximation, but this may not be very obvious from outside.

fweber144 commented 10 months ago

Actually, I was looking for some output element of the submodel fits returned by fit_gam_callback() and fit_gamm_callback() that indicates convergence. So for GAMs, I would need approval that https://github.com/stan-dev/projpred/blob/97c5bea9cafd041c8b2de4dd1b3aa56a97e3e469/R/divergence_minimizers.R#L1022-L1023 is correct and for GAMMs, I wonder where the same elements from gamm_fit_s$gam are (where gamm_fit_s indicates an object returned by fit_gamm_callback()). Implementing own diagnostics is not a bad idea, but usually, there is always a convergence indicator somewhere in the output object (so we wouldn't have to implement new diagnostics ourselves).

AlejandroCatalinaF commented 10 months ago

I believe that line is okay, but I haven't worked with that package for some time, so I'm afraid I can give you much insight regarding their internals.

fweber144 commented 10 months ago

No problem, I understand. Then I guess someone needs to inspect this in detail in the future.