search

ktw5691 / psychtm

Text Mining for Psychological Research
Other
9 stars 1 forks source link

Compute WAIC for model comparison #11

Closed ktw5691 closed 4 years ago

ktw5691 commented 5 years ago

Complete for sLDA-X with binary outcome. Need to add for LDA and sLDA/sLDA-X with continuous outcome and sLDA with binary outcome.

ktw5691 commented 5 years ago

Complete for sLDA with binary and continuous outcomes. Complete for regression with no text for binary and continuous outcomes. Still need to support for LDA.

ktw5691 commented 5 years ago

Following Merkle et al. (2019) in Psychometrika, the marginal WAIC should probably be the default used for model comparison while the conditional WAIC could be another alternative. Currently, the conditional WAIC is computed and the marginal WAIC is not yet implemented. Merkle et al. (2019) detail an importance sampling adaptive quadrature approach to do this with continuous latent variables. Would this be appropriate in the discrete latent variable (i.e., topic) case?

ktw5691 commented 5 years ago

Merkle et al. (2019) detail an importance sampling adaptive quadrature approach to do this with continuous latent variables. Would this be appropriate in the discrete latent variable (i.e., topic) case?

We should be able to directly sum over the discrete latent variables in the joint posterior samples (no quadrature/sampling methods needed).