Both reviewers were intrigued by this study and suggest that the authors should be given the opportunity to revise and resubmit it. However, they both indicate significant concerns about the framing and presentation of the results. In particular, the authors will need to spend time thinking about the "why" questions around their conclusions. Why are unidimensional measures failing, why is a multidimensional measure better?
R1:
2) Rework the abstract, intro and paper as a whole to make clear what the take-away message should be. To make the paper interesting to a wider audience beyond the two sets of authors who seem to be deeply engaged in the topic (e.g. Claassen and colleagues and Solt and colleagues) it needs a clearer message. That message, in my view, needs to be more than “other people’s unidimensional measures don’t correlate with things with that we think might relate to democracy.” Rather, I think the paper needs to explain (or at least speculate) why they don’t correlate — e.g., perhaps unidimensional measures are often driven by questions that ask about support for democracy in the abstract, without defining what it is (see point 1). Abstract support for democracy, however, does not correlate with support for items that actually relate to support for features of liberal democracy. Alternatively, it could be people have a different understanding of the term “democracy, even if their understandings do relate to some characteristics of liberal democracy. They even may agree on the basic items by weigh them differently. Or it could, of course, be that democracy as a term is meaningless in the context of a survey, and therefore the “Churchill” style questions, which are most commonly asked in a comparative manner are actually rather meaningless (or at least not related to support for things that we actually associate with democracy). Which of these explanations is most likely behind the lack of a correlation? This needs to be clear from the beginning of the paper.
3) Depending on the answers to points 1 and 2, I would have been very interested in questions such as: does region matter? Or are there countries or regions where the understanding of liberal democracy is clearer and more closely associated with responses to the Churchill style questions (or the unidimensional measure)? Does it work better in the West than outside of the West? Has that changed over time or with the advent of authoritarian politicians in Europe? I don’t think that the paper necessarily needs to answer all these questions, but the answers would help us to understand why the correlation doesn’t exist (or where it does exist) and help address the first two points.
[ ] More discussion about the why (with lit rather than analysis, I guess?)
Lu & Chu 2021 could be a good source
R2:
Third, could the weak correlation between the latent variable measure and other questions on democracy be explained by the fact that the measure relies on different questions in different countries?
[ ] Better explanation about how IRT works beyond the exact question format
Editor:
R1:
R2: