Rm analyses that shouldn't be run (BT random effects models)

[egouldo]

Hey Tim,

I think this may potentially be a result of the way I repeated the analyses over each of the different data subsets, i'll look into it.

So, the models may have been fitted automatically, but I never extracted the results of the cases where we had < 5 analyses with random effects included.

1. I’ll make sure these cases are removed from the manuscript first,
2. And then can remove these from the analysis pipeline.

Elliot.

On 1 Jun 2024, at 6:14 am, Tim Parker parkerth@whitman.edu wrote:

Hi Elliot,

In the supplement C.6.1 Effect Sizes Zr also C.6.2 Out of sample predictions yi

It appears that the presence of mixed effects (whether or not the analyst included a random effect) was included in both the BT and the Euc models, but I thought that we only included this effect for Euc since there were too few BT analysts that did not include a random effect.

-Tim

[egouldo]

Sections C.6.1 and C.6.2 refer to the multivariate models for both Zr and yi, not the deviation from the analytic mean explained by inclusion of random effect models.

• [x] Were we supposed to remove this fixed term from the multivariate models if we didn’t meet the preregistered threshold?

I may have misinterpreted the original preregistration if so.

Depending on Tim's response I’ll update the code or leave as is.

[egouldo]

From Tim:

Got it. Yes, the registration stated that we would include the 'random effect presence' variable in the univariate and the multivariate analysis only if it crossed the threshold:

"Thus if there are at least 5 analyses that do and 5 analyses that do not include random effects, we will add a binary predictor variable “random effects included (yes/no)” to our set of univariate analyses and will add this predictor variable to our multivariate model described below".

• [ ] @egouldo to remove predictor term from models and re-run
• [ ] Check random effects need to add effectID, we said we would run with analysis ID, but not included in reported model.
• [ ] Check convergence / singularity, if fails, then return back to ReviewerID only.