brmsfit cumulative model & Latent projection predictive feature selection

[GLad-Arioch]

Hi,

I try to use the varsel / cv_varsel functions of the prejprod package with brmsfit objects.

The models are multilevel (3 levels) ordinal cumulative models.

Models set-up includes the brms option: “save_pars = save_pars( all = TRUE )”

Their only peculiarity, if any, is the use of the monotonic effect function mo() (Bürkner& Charpentier 2020) to specify one of the regressors.

In consequence, I add the “latent = TRUE” option to the get_refmodel() function or directly to the varsel() / cv_varsel() functions.

The following lines of codes return a “Projpred does not support distributional models” error message.

library(tidyverse) library(brms) library(projpred)

fit4 <- readRDS( “fit4.RDS” )

fit4ref <- get_refmodel( fit4 , latent = TRUE )

or alternatively

fit4_vscv <- cv_varsel( fit4 , latent = TRUE )

What are my mistakes?

Best regards,

Guillaume

[fweber144]

Hi Guillaume,

Currently, projpred does not support monotonic effects, see "Supported types of models" (paragraph "On the side of the predictors, [...]").

Perhaps the error message could be more informative. Tagging @paul-buerkner so he knows about it (because this error is thrown by brms:::get_refmodel.brmsfit()).

[GLad-Arioch]

Thank you, Frank, for your quick answer.

I re-ran the models with simple factors ( dummy coding) instead of monotonic effects.

fit4ref <- get_refmodel( fit4 , latent = TRUE ) or alternatively fit4_vscv <- cv_varsel( fit4 , latent = TRUE )

return the same error as before : “Projpred does not support distributional models”.

Here is the specification:

fit4 = brm(data = datacom, family = cumulative(), prior = c(prior(normal(0, 1), class = b), prior(normal(0,3), class = b, dpar = disc)), bf(state6class ~ 1 + z_weeknumber + I(z_weeknumber^2)

• magn8cat (formerly mo(mag8class))
• a bunch of predictors, including categorical * continuous interactions
• (1 + z_weeknumber + I(z_weeknumber^2) | regid/unitid))
  • lf(disc ~ 0 + type, cmc = FALSE), save_pars = save_pars(all = TRUE), silent = 0, iter = 2000, warmup = 1000, chains = 8, cores = 8, control=list(adapt_delta = 0.97), seed = 1313, backend = "cmdstanr")

I now suspect that unequal variances parameters (disc / discrimination; see Bürkner& Vuorre 2019 ), in the lf() formula are the causes. Unequal variances terms improve notably the specification and are "theoretically" sound. I want to keep them. I will use Projpred on models of other kinds. I hope this information helps improve the amazing software you are developing.

Best regards,

Guillaume

[fweber144]

I now suspect that unequal variances parameters (disc / discrimination; see Bürkner& Vuorre 2019 ), in the lf() formula are the causes.

Yes, given your lf(disc ~ 0 + type, cmc = FALSE) line, you do indeed have a distributional model. So the error message was correct and I'll close this issue here. Thanks for the follow-up.

@paul-buerkner: No need to change anything in brms, I guess.