search

fsolt / dcpo_gender_roles

Comparable, cross-national time-series estimates of public opinion toward egalitarian gender roles
2 stars 0 forks source link

Explanation of which survey questions used and how they are distributed #27

Closed byngdeuk closed 2 years ago

byngdeuk commented 3 years ago

Reviewer #1

  1. Although they are pulling together 46 survey items from 88 datasets, it is not clear how much the items overlap. My guess is that for some countries and years, there are multiple items that are used to generate the estimate, while for others, there may be only one. For the 2695 country-years for which they provide estimates, how often were they based on one or two items?

Reviewer #2

  1. I think it would also add a lot to the paper to give an indication in the paper (rather than an appendix) of (a) what kinds of survey questions are used in the model (at the bottom of p4 I wasn't sure if the inclusion of women in higher office was an indicator or a validation) and (b) what kinds of questions you are able to use for bridging (or if you do not use bridging) / what kind of overlap you have between surveys. This is useful both for people reading the paper for the methods (I wondered both of these things repeatedly while reading the paper and the Appendix only kind of helped) and for the substantive scholar who wants to understand fully how the measure is constructed.
fsolt commented 2 years ago

Call this overlap in the figure caption

fsolt commented 2 years ago

Why is overlap important? Overlap means we get better estimates of dispersion and difficulty (and theta!); non-overlap means we get more coverage of country-years

Cassandra: more items doesn't guarantee good estimates of theta

need to cite to the DCPO article's k-fold cross-validation to show that one observation does a good job of predicting

Is observation a good word? It's not "one guy"--it's a survey of a country-year

Even a single observation is a lot of information: it's ~1000 individual respondents

What information is Figure A2 providing, really?

Consider dropping the number of joint observations in A2, just dichotomize