Political identification survey

[JamesPHoughton]

We want to measure participants' party identifications. This is a survey that we will probably use many times in lots of different experiments, and so needs to be bulletproof. We should see what is done in ANES, PEW, GSS, and possibly other surveys (Census?) and arrive at the least controversial way of asking this set of questions.

I expect we will ask something about both party id (republican/democrat) and ideology (liberal/conservative). This may or may not be a unidimensional scale, i.e. we could have a different scale for "how much do you identify with the republican party"? "With the democratic party"? "As a libertarian"? etc.

Let's use this issue to identify alternative presentations of the question and identify which one we are going to use and why.

@markwhiting, @linneagandhi, @TutiGomoka, do you know if we already have something like this in the turkers panel?

[TutiGomoka]

Hi @JamesPHoughton; As Mark mentioned, the current way we phrase questions in our demographics instrument is as is shown here.

Our basis for the ideology question is this Rand Paper: See page 1869 footnote of [https://psycnet-apa-org.proxy.library.upenn.edu/fulltext/2018-46919-001.pdf] Our basis for the partisanship question is this paper: https://www.pnas.org/doi/10.1073/pnas.2114388118#supplementary-materials

[isaacgat]

Hi Tuti, I have a few questions about the survey.

1. I notice our survey question about partisan split is coded between Democrat, Republican, Independent, and Other, but I am curious if this is a collapsed variable (were you measuring along a greater scale, e.g., 1-7?) that could tell us more about the independent leaners. If not, we should rethink how we gather this information because the label "independent" is not as informative as we might like. A lot of Americans are independents but only a small portion of those are pure independents. We need to measure partisan lean because it is just as predictive as pure partisan split (see Myth 2).
2. It looks like the first paper asks participants to identify ideology on a 5-point Likert scale, whereas the second paper asks participants to identify partisanship on a 7-point scale (see codebook). We should probably standardize this and make these scales similar.

[TutiGomoka]

@isaacgat

1. I'm looking at the supplementary materials for the second paper (see screen-grab below). It looks like they asked people “Generally speaking, do you consider yourself a...” and it was scale of 1 - 4 of democrat, republican, independent and other party. So, my understanding is that its a 4-point scale. Concerning the "independent" option, we chose this model of asking questions because it seems like the most simple and most used in major survey instruments such as Pew Research. Even though breaking the independent choice into multiple options might be more informative for us, we also need to consider how respondents understand these questions and how likely they are to respond accurately to questions which seem confusing to them. For example, a regular person might have good understanding of what an "independent" is because it's the way they are currently registered to vote &/or fill out surveys elsewhere. But might be confused as to what an "independent republican leaning" is. And so in survey instruments like this we try to avoid such kind of newness/ambiguity to prevent any interference of our results. This is why we based of this question on the most recent academic standard (David Rand 2021) who does a lot of research work related to the topics we study. Perhaps @markwhiting can also weigh on this a bit.

3. We personally don't see a problem in different questions having different likert scales because they are asking different things. To my understanding the partisanship question has a 4-point likert scale (see my answer to the first question above).

[markwhiting]

@isaacgat it depends a bit on which model of political identity you adhere to, and our approach is in part intended to cover several bases in that regard. The party identification question is more around how you register, or would register, however, there's absolutely room to ask something more nuanced, but I'd probably do that in addition to what we have now, as opposed to instead. i.e., a secondary question that asks for leaning specifically.

We use the 5 point version for both fiscal and social in our demographics instrument, but we also normalize these for standardization across other data sources that we have inducted into our panel in a relatively standard approach.

We're always interested to improve our panel instruments as they are used by several groups in the lab. So if you find a lean question that we could add in here, that might be very useful.

[isaacgat]

@markwhiting Thank you for clarifying! I see how having data about political affiliation w.r.t. voting is valuable. I had a conservation with James and Tuti before lunch about the question structure and we seemed to agree that the current question is part of a larger question branch (per the screengrab that Tuti sent) that we may consider gathering from future participants for our study. In our case, we are highly interested in the lean of our respondents because we want to test the measurable impact of the deliberation on people across the broad spectrum of political backgrounds. So the lean questions that we might add are follow-ups that are included by Rand in his question documentation. I am currently researching the breakdown of these spectrum Likert scale questions into more digestible questions and then summarizing the variables ex post facto (like Rand and the ANES seem to do). If there appears to be an agreed-upon way of structuring these questions, I will note those findings here.

[markwhiting]

Great. Just as a side note, basically everyone in our panel has done or will do our demographic instrument, and we can adjust that instrument to include other things that we want from everyone. However, if there are really particular things you need from people it might be useful to also have a supplementary instrument for that purpose (data from which we would pull into our panel data when possible too).

[TutiGomoka]

@markwhiting; One thing that Isaac mentioned in our conversation that I found interesting is that the results/scale recorded for the partisanship question in that Rand paper was sort of derived from that first political registration question + the follow up leaning questions to = a 7-point Likert scale in the results. Whereas right now in the panel we just have our political party responses recorded as democrat, republican, neutral etc. Is that something we want to think about? re: derive result based on branches of different questions?

[markwhiting]

One of the major criticisms of doing this is that it assumes that independents are between two extremes (republicans and democrats). I think the standard argument for moving beyond this is to ask political leaning questions, e.g., the fiscal and social questions we have, so that people's identity can be mapped in a higher dimensional space.

Again, I think there remains a fair amount of disagreement in the literature about the best possible way to do this, but I think the questions we ask let us explore that space reasonably well.

I'd probably lean away from trying to treat political lean as a single dimension, and instead use some of the more nuanced approaches some people take. Josh Ludan, someone who works (or used to work) with Homa may have done a bit more digging here and have a more nuanced awareness of the latest suggestions.

[isaacgat]

Hi @markwhiting and @TutiGomoka,

James and I have agreed that we should gather the full scope of the partisan divide in the two-question Likert question per the Rand survey. Although the literature is split on whether or not the two-question Likert scale is superior, it appears that Rand's full metric is preferable because it gives us a larger pool of partisans to pick from as we can include leaners in our analysis. Furthermore, Levendusky (2018) uses a single-question 7-point Likert scale to measure ideology breakdown while using the tiered responses to measure the Republican-Democrat in the same survey. My gut feeling is that this is appropriate given that we think of the left-right as a spectrum while the Republican-Democrat split is less linear. In any event, this reassures us that we are good to use single-part Likert questions to measure ideology (separately for both fiscal and social) while using a two-part Likert question to measure partisan divide as we have set up; mixing and matching types has been done successfully before.

@JamesPHoughton has thoughts on how to collect this additional party ID data that he will comment on here.

[markwhiting]

Sounds great.

[JamesPHoughton]

Next step for this is to add a question to the basic panel survey? @TutiGomoka how would we do that?

[JamesPHoughton]

For the US, added this standard survey: https://github.com/Watts-Lab/surveys/tree/main/surveys/politicalPartyUS