easystats / report

:scroll: :tada: Automated reporting of objects in R
https://easystats.github.io/report/
Other
694 stars 70 forks source link

Report brms refitting models #374

Open jgeller112 opened 1 year ago

jgeller112 commented 1 year ago

Maybe I am missing something, but is there a way to avoid always refitting brms models using the report function? I have pretty large models and using report will either error out or run forever.

strengejacke commented 1 year ago

@rempsyc @mattansb @DominiqueMakowski I'm not sure, but I guess this is due to computing Bayes factors when retrieving the performance-table? We should try to find the bottle neck here, maybe using the profile package.

@jgeller112 do you have a model you can share, so we can try to reproduce your issue? Else, maybe you find a model out of our pre-compiled models from here: https://easystats.github.io/circus/reference/index.html and just name one of those example models that behaves similarly.

mattansb commented 1 year ago

I think this is because report standardizes coefficients using the refit method (which it 100% should not do since it does not adjust the priors accordingly).

jgeller112 commented 1 year ago

Here you go:

library(tidyverse)
library(report)
fit_c <- read_rds("https://osf.io/hmqpc/download")
report(fit_c)
strengejacke commented 1 year ago

If I understand the profiling right, one issue is logLik():

image

And another bottle neck is rstan::summary():

image

Given that already the simple call to summary() takes ~ 7 seconds on my machine, this looks like the issue is due to the very complex model. The profile-file is attached (I used profvis), maybe we find ways to remove some of those time-consuming things.

report_brms.zip

DominiqueMakowski commented 1 year ago

I think this is because report standardizes coefficients using the refit method (which it 100% should not do since it does not adjust the priors accordingly).

yes, the standardization method should be changeable to something else than "refit" (?)

strengejacke commented 1 year ago

@jgeller112 Can you then try to run report(fit_c, include_effectsize = FALSE)?

jgeller112 commented 1 year ago

I get

Error in report_table.brmsfit(x, include_effectsize = FALSE, ...) : 
  formal argument "include_effectsize" matched by multiple actual arguments