search

drizopoulos / GLMMadaptive

GLMMs with adaptive Gaussian quadrature
https://drizopoulos.github.io/GLMMadaptive/
60 stars 14 forks source link

Discussion: Model Selection with GLMMadaptive #30

Closed mamscience closed 4 years ago

mamscience commented 4 years ago

Dear dr. Rizopoulos

Thanks for writing GLMMadaptive. It's been intuitive to work with, even for somebody who's new to GLMM like myself.

My question is regarding model selection: What approach would you advice in model selection? In your vignette Goodness of Fit you suggest using DHARMa to aid in the selection. With binomial models, only the residuals vs fitted graph is useful to assess fitness. I believe that this is hard, particularly when all models do not fit perfectly. Can you suggest more tools or approaches for selecting models?

Kind regards, Michel

drizopoulos commented 4 years ago

Dear Michel,

If you want to compare between models (i.e., this is what I understand with “model selection”), you can use a likelihood ratio test if the models are nested or look at the information criteria AIC and BIC if the models are not nested. Both are available in the anova() function.

Best, Dimitris

From: mamscience notifications@github.com Sent: Wednesday, November 4, 2020 11:15 AM To: drizopoulos/GLMMadaptive GLMMadaptive@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: [drizopoulos/GLMMadaptive] Discussion: Model Selection with GLMMadaptive (#30)

Dear dr. Rizopoulos

Thanks for writing GLMMadaptive. It's been intuitive to work with, even for somebody who's new to GLMM like myself.

My question is regarding model selection: What approach would you advice in model selection? In your vignette Goodness of Fit you suggest using DHARMa to aid in the selection. With binomial models, only the residuals vs fitted graph is useful to assess fitness. I believe that this is hard, particularly when all models do not fit perfectly. Can you suggest more tools or approaches for selecting models?

Kind regards, Michel

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdrizopoulos%2FGLMMadaptive%2Fissues%2F30&data=04%7C01%7Cd.rizopoulos%40erasmusmc.nl%7Cdc0b61d9fcaa42688a3908d880aa7649%7C526638ba6af34b0fa532a1a511f4ac80%7C0%7C0%7C637400816899356906%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=S8FrULaawQmSgOqRXERWdsAoN6o2FhWi5tjK7p7II7o%3D&reserved=0, or unsubscribehttps://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FADE7TT4SD6QBGUZCBVD5LV3SOESRLANCNFSM4TJ3JSEA&data=04%7C01%7Cd.rizopoulos%40erasmusmc.nl%7Cdc0b61d9fcaa42688a3908d880aa7649%7C526638ba6af34b0fa532a1a511f4ac80%7C0%7C0%7C637400816899366900%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Aya5UTIdTOwKY%2Fs0AAovRCFpibjaOPrEmCH1FW9LTmI%3D&reserved=0.

mamscience commented 4 years ago

Thanks for your reply. I've read in multiple articles saying that using AIC/BIC should be used with caution with GLMM, but I understand from your reply that this is not the case for non-nested models.

I had a follow-up question: How would you choose between a model that has a better AIC fit but a worse resid vs fitted plot, compared to a model that has a worse AIC but better resid/fitted plot?

drizopoulos commented 4 years ago

This question is more about statistics rather than about the functionality of the package.