Do you know the Wooly Bully? Testing era-based knowledge to verify participant age online

[ben-domingue]

https://osf.io/bn4xy/?view_only=7252e963f3bd4c0c981eed6ddd085ee8 paper. https://link.springer.com/article/10.3758/s13428-022-01944-y#Sec34

[ben-domingue]

@anniehang is working on this

[anniehang]

I think representing the subject's choice for each item with numerical values is not meaningful in practice. Should I still use the numerical values as the response, or should I use correct(1) or incorrect(0) as the response?

[ben-domingue]

corerct/incorrect would be great!

[anniehang]

ok!

[anniehang]

vertifyage_study1a.csv vertifyage_study1b.csv vertifyage_study2.csv vertifyage_study3a.csv vertifyage_study3b.csv vertifyage_study3c.csv

[ben-domingue]

morning @anniehang. this is great!

i am wondering if we could perhaps collapse studies 1b, 3a, 3b, and 3c. they seem to be based on a common version of the measure (best i can tell, we can check this) and we can add an 'age' column and then also a 'group' column (with values: 1b, 3a, 3b, 3c). what do you think? my general hope is to have relatively few datasets coming from each study and to the extent that we just have different people taking the same measure here i'm ok with that (but it would be good for us to verify that they are taking the same measure; they seem to be from my skim of it but do you have add'l insight?).

[ben-domingue]

also we should make 'i dont know' NA if we haven't already

[anniehang]

Ok, I think that's a good idea. The number of participants reported in the paper reflects the count after missing values were removed. so I directly eliminated participants with missing data from the analysis.

[ben-domingue]

do you want to put together an updated dataset and i can take a look at it? and, just to confirm, the 'don't know' responses were already reported as NA values?

[anniehang]

ok, i got it!

[anniehang]

vertifyage_study1a1b3b3c.csv vertifyage_study2.csv vertifyage_study3a.csv

[ben-domingue]

one question about 1a1b3b3c: there are only 18 items but it looks (from the text) like there should be 19. did one get dropped for some reason?

[anniehang]

Oh, I made a mistake; there are 19 items in 1a. I think I should merge 1a and 2 together. However, what confuses me is that the data for 1b, 3b, and 3c only provide 18 items.

[ben-domingue]

hm. very strange. if you think that something may be wrong in their data (such that it doesn't map onto what they say), i can email them? i'm sure they'd be happy to know that there is a problem if we are confused.

[anniehang]

ok, I can send them an email to inquire about it.

[ben-domingue]

i'm also happy to email them! let me know if you prefer that? no big deal for me to do it :)

[anniehang]

If it's not too much trouble, I would be happy if you could email them. 😃

[ben-domingue]

happy to :)

[ben-domingue]

second email sent july 2. @ben-domingue

[ben-domingue]

They finally got back to me: Thanks for reaching out and for your interest in our paper! I’m actually seeing 19 in some and 20 in others, with the 20th extra item being “Tik Tok,” which didn’t make it into our final list of 19 (it was in the original development of the items, which you can see in the supplemental materials). I can’t say why it was dropped in some of the studies since data collection was a long time ago (and I wasn’t involved yet). If you decide to include the items in the IRW, you should include the 19 that are listed in Table S1. What I would propose is that we just subset to that final list of 19 if that is easy?

[ben-domingue]

@anniehang see above

[anniehang]

ok, i got it !

[ben-domingue]

thanks annie. you are the best :)

[anniehang]

Apologies! I realized my previous mistake. Some sub-datasets did not provide answers for Q8, while others had increased answers for the 'TikTok' column, so I mixed them up. By reviewing the participants' original answers to Q8, I can infer whether their responses were correct. Now each sub-experiment has 19 items.

[anniehang]

AVI-S.csv

[ben-domingue]

great!!