Collapsing across models

[wibeasley]

@ampiccinin, tomorrow/Wednesday when we talk, @andkov and I would like to show you the variability of estimates of the same physical process --when collapsing/averaging/aggregating across the ~11 cognitive processes.

In each of these screenshots below, one row is one run through Mplus.

Also, consider how you want to collapse for stats other than est. I'm taking the median, which might be better when collapsing across the 11 different p-values?

point estimate examples

p-value examples

[ampiccinin]

Wow! that is very surprising - and a bit disconcerting.

Also interesting that the largest value for grip (89.9) has a p-value nowhere near 0.05

Although the n appears the same all the way through, these estimate differences seem large (I wonder about the SEs). On the other hand, only one intercept (for FAS - if I am reading this correctly) has a p-value below 0.05, so that may just be a fluke. Clearly, as with just about everything, it just keeps getting more interesting the deeper we dig...

Seems like averaging may be less defensible...

[wibeasley]

Cool, I'm happy to talk about it as much as you'd like. @andkov prodded me to create a dynamic table for this, so you guys could look at in, without me setting a breaking point in the code. I'll try to finsih that tonight.

One thing --we were wandering about when I got those screenshots. Noticed the coefficient column is different between the tables. For instance, the '00' in the top table represents $gamma_00$.

[wibeasley]

@andkov & @ampiccinin This table has a few filters on it already, so it won't clog your computer with r scales::comma(nrow(ds_long)) rows. It's only model_type=="aehplus" & subgroup=="female" & process_a=="gait". Otherwise the browser was dying and the html file was 13MB. Th dynamic table was as interactive as a turtle.

I fixed a small problem, so the numbers don't match the screenshot above, but the concept remains. The 11 row above the red line correspond to gait (remember this whole table is gait). The 11 rows below it correspond to process_b. I drew four little horseshoes that illustrate types of discrepancies that we talked about hypothetically last time. Now after Andrey's nudging, they're not so hidden or hypothetical.

If you're brave enough to load this big report through the web, use https://rawgit.com/IALSA/IALSA-2015-Portland/master/reports/growth-curve-1/growth-curve-1.html. It's probably faster if you load it locally though.