search

willschulz / bad-influence

Video discussion experiments to counter pathological influence in political discussions
0 stars 0 forks source link

Reorient discussion questions to be about preferences for policy to move left/right #1

Closed willschulz closed 1 year ago

willschulz commented 1 year ago

Proposed changes to questions:

Original: "What is your position on policing?" (Multiple-Choice-Response from de-fund the police to crack down on crime) Proposed revision: "Overall, do you feel positively or negatively about the police?" (left-right slider response)

Original: "Thinking about police departments in your area, do you think that spending on policing should be... (MCR from increased a lot to decreased a lot)" Proposed revision: "Police should be involved in more or fewer aspects of public life/safety?" (left-right slider response)

Original: "Racism is a problem in policing." (MCR from strongly agree to strongly disagree) Proposed revision: Same question, but with a left-right slider response format.

Original: "Most of the time, when the police use force against civilians it is justified/for the greater good." (MCR strongly agree to strongly disagree) Proposed revision: "Should it be made easier or harder for police to use force against civilians?" (left-right slider response)

RachelAbigail commented 1 year ago

Really like these. I would slightly tweak the last three so the phrasing/formatting is consistent between all four.

  1. Overall, do you feel positively or negatively about the police?
  2. Do you believe police should be involved in greater or fewer aspects of public life?
  3. To what extent do you believe racism is a problem in policing?
  4. Do you believe it should be made easier or harder for police to use force against civilians?
JamesPHoughton commented 1 year ago

I don't have sliders implemented for these types of questions yet, I'm afraid. We'd need to use a Likert scale for now.

One thought I had was that if we want to eventually expand to questions about other topics, possibly with variations on topics already proposed in ANES or other social survey, it may make sense to have at least one of these questions (maybe the first?) come almost directly from ANES. Then we can say that we chose the questions from this source, and expanded on them by making them more specific and concrete.

Two questions from ANES that discuss policing that are already in my data source are:

Prompt

How often do you think police officers use more force than is necessary?

Responses

  • Never
  • Rarely
  • About half the time
  • Most of the time
  • All the time

And

Prompt

What is the best way to deal with the problem of urban unrest and rioting? Some say it is more important to use all available force to maintain law and order, no matter what results. Others say it is more important to correct the problems of racism and police violence that give rise to the disturbances. And, of course, other people have opinions in between.

  • Where would you place yourself on this scale?

Responses

  • 1 (Solve problems of racism and police violence)
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7 (Use all available force to maintain law and order)
RachelAbigail commented 1 year ago

Perhaps sub in the ANES use of force question for the one we have now, and add the urban unrest question? The urban unrest Q doesn't really overlap with anything we have now so it adds a new dimension to the discussion topic, and it was one of those high variance, high polarization questions on James' map.

dcknox commented 1 year ago

To @JamesPHoughton 's points, in addition to being already implemented, Likert would keep us consistent with ANES as well. So let's go with that.

Aligning them to all be some variant of "do you believe..." per @RachelAbigail 's suggestion seems like a good idea.

The left side of the urban unrest question ("solve problems of racism and police violence") overlaps with the current #3 proposal. I'm open to including both, but if we have to pick one then I'd have a preference for sticking closer to ANES.

xehu commented 1 year ago

I actually feel like the second ANES question feels like a mix of our original #3 and #4. The left side is similar to #3, while the right side is similar to #4.

Attempting to combine options from Rachel, James, and Dean (and making minor wording tweaks):

  1. Overall, how positively or negatively do you feel about the police?
  2. Do you believe police should be involved in greater or fewer aspects of public life?
  3. How often do you think police officers use more force than is necessary?
  4. To what extent do you believe racism is a problem in policing?
  5. What is the best way to deal with the problem of urban unrest and rioting? Some say it is more important to use all available force to maintain law and order, no matter what results. Others say it is more important to correct the problems of racism and police violence that give rise to the disturbances. And, of course, other people have opinions in between. Where would you place yourself on this scale?

3 from ANES replaces the following:

Although I feel like this ANES question is slightly problematic, as it reminds me of one of our original questions --- it seems to ask people their perceptions on what is in fact a statistical questions. I feel like I would answer this question with something like, "I don't know, I'd want to look at the numbers" --- because even though cases like George Floyd stick in my memory, I don't actually know the stats for how often that kind of unnecessary force occurs in general. In this respect, I think our question is slightly better.

dcknox commented 1 year ago

We could focus the question to bring it closer to our original intent while still keeping it tied to the ANES. What about something like "Do you believe officers should be able to use all available force to maintain law and order, or is it more important to reduce police violence?"

Get Outlook for Androidhttps://aka.ms/AAb9ysg

From: Xinlan Emily Hu @.> Sent: Wednesday, February 22, 2023, 1:22 PM To: willschulz/bad-influence @.> Cc: Knox, Dean @.>; Assign @.> Subject: Re: [willschulz/bad-influence] Reorient discussion questions to be about preferences for policy to move left/right (Issue #1)

I actually feel like the second ANES question feels like a mix of our original #3 and #4. The left side is similar to #3, while the right side is similar to #4.

Attempting to combine options from Rachel, James, and Dean (and making minor wording tweaks):

  1. Overall, how positively or negatively do you feel about the police?
  2. Do you believe police should be involved in greater or fewer aspects of public life?
  3. How often do you think police officers use more force than is necessary?
  4. To what extent do you believe racism is a problem in policing?
  5. What is the best way to deal with the problem of urban unrest and rioting? Some say it is more important to use all available force to maintain law and order, no matter what results. Others say it is more important to correct the problems of racism and police violence that give rise to the disturbances. And, of course, other people have opinions in between. Where would you place yourself on this scale?

3 from ANES replaces the following:

Although I feel like this ANES question is slightly problematic, as it reminds me of one of our original questions --- it seems to ask people their perceptions on what is in fact a statistical questions. I feel like I would answer this question with something like, "I don't know, I'd want to look at the numbers" --- because even though cases like George Floyd stick in my memory, I don't actually know the stats for how often that kind of unnecessary force occurs in general. In this respect, I think our question is slightly better.

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/willschulz/bad-influence/issues/1*issuecomment-1440562094__;Iw!!IBzWLUs!W-zsUD6lfsR82tbOXvDCm40WRiNoW8q6ZwaSH33zV4izq99Li6UhcPrjNz3KponQHUGLb18DCW0BEUn0TBuKEbP6$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AB44CXVAYAYQ7OBYJ23DPX3WYZKPBANCNFSM6AAAAAAVEPZYMI__;!!IBzWLUs!W-zsUD6lfsR82tbOXvDCm40WRiNoW8q6ZwaSH33zV4izq99Li6UhcPrjNz3KponQHUGLb18DCW0BEUn0TIAzDU7i$. You are receiving this because you were assigned.Message ID: @.***>

xehu commented 1 year ago

Hmm... thinking about this more, the following 4 questions sound similar to me, as all of them are different ways of getting at whether police use of force is justified or not. I also think that each one has their pros and cons, which I briefly discuss below.

a. (ANES) How often do you think police officers use more force than is necessary? [See previous comment: I feel like this question is too statistical.] b. (ANES) What is the best way to deal with the problem of urban unrest and rioting? Some say it is more important to use all available force to maintain law and order, no matter what results. Others say it is more important to correct the problems of racism and police violence that give rise to the disturbances. And, of course, other people have opinions in between. Where would you place yourself on this scale? [On second thought, I feel like this question is weird because it's triple-barrelled: it combines a question about police use of force with a question about racism and a question about urban unrest/rioting. I think the question assumes a false binary, that the solution to rioting is either "police use of force" or "solve racism;" even though "people have opinions in between," I wonder whether we can measure those in-between opinions accurately with this question.] c. (Will's suggested question) Do you believe it should be made easier or harder for police to use force against civilians? [I like this; it's more policy-focused.] d. (Dean's suggested question) Do you believe officers should be able to use all available force to maintain law and order, or is it more important to reduce police violence? [I also like this; it's more focused on force, without wrapping it up with other issues, like racism or urban unrest.]

Perhaps we choose (d) — Dean's suggestion — and add separate questions to get at topics like urban unrest? For example:

"An important contributing factor to urban unrest and rioting is lack of policing." [Agree-Disagree] "An important contributing factor to urban unrest and rioting is racism in policing." [Agree-Disagree]

On Wed, Feb 22, 2023 at 4:31 PM Dean Knox @.***> wrote:

We could focus the question to bring it closer to our original intent while still keeping it tied to the ANES. What about something like "Do you believe officers should be able to use all available force to maintain law and order, or is it more important to reduce police violence?"

Get Outlook for Androidhttps://aka.ms/AAb9ysg

From: Xinlan Emily Hu @.> Sent: Wednesday, February 22, 2023, 1:22 PM To: willschulz/bad-influence @.> Cc: Knox, Dean @.>; Assign @.> Subject: Re: [willschulz/bad-influence] Reorient discussion questions to be about preferences for policy to move left/right (Issue #1)

I actually feel like the second ANES question feels like a mix of our original #3 and #4. The left side is similar to #3, while the right side is similar to #4.

Attempting to combine options from Rachel, James, and Dean (and making minor wording tweaks):

  1. Overall, how positively or negatively do you feel about the police?
  2. Do you believe police should be involved in greater or fewer aspects of public life?
  3. How often do you think police officers use more force than is necessary?
  4. To what extent do you believe racism is a problem in policing?
  5. What is the best way to deal with the problem of urban unrest and rioting? Some say it is more important to use all available force to maintain law and order, no matter what results. Others say it is more important to correct the problems of racism and police violence that give rise to the disturbances. And, of course, other people have opinions in between. Where would you place yourself on this scale?

3 from ANES replaces the following:

  • Do you believe it should be made easier or harder for police to use force against civilians?

Although I feel like this ANES question is slightly problematic, as it reminds me of one of our original questions --- it seems to ask people their perceptions on what is in fact a statistical questions. I feel like I would answer this question with something like, "I don't know, I'd want to look at the numbers" --- because even though cases like George Floyd stick in my memory, I don't actually know the stats for how often that kind of unnecessary force occurs in general. In this respect, I think our question is slightly better.

— Reply to this email directly, view it on GitHub< https://urldefense.com/v3/__https://github.com/willschulz/bad-influence/issues/1*issuecomment-1440562094__;Iw!!IBzWLUs!W-zsUD6lfsR82tbOXvDCm40WRiNoW8q6ZwaSH33zV4izq99Li6UhcPrjNz3KponQHUGLb18DCW0BEUn0TBuKEbP6$>, or unsubscribe< https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AB44CXVAYAYQ7OBYJ23DPX3WYZKPBANCNFSM6AAAAAAVEPZYMI__;!!IBzWLUs!W-zsUD6lfsR82tbOXvDCm40WRiNoW8q6ZwaSH33zV4izq99Li6UhcPrjNz3KponQHUGLb18DCW0BEUn0TIAzDU7i$

. You are receiving this because you were assigned.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/willschulz/bad-influence/issues/1#issuecomment-1440828198, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG3VWKPS3EY2YRWVZNIVKYDWY2ASBANCNFSM6AAAAAAVEPZYMI . You are receiving this because you were assigned.Message ID: @.***>

willschulz commented 1 year ago

I like @RachelAbigail's suggestions a lot, and I'm open to @dcknox's "focused" version of the ANES Q, though I find "all available force" ambiguous and sinister (does this mean shooting people for jaywalking?). I worry that this will lead to discussion of different scenarios in different groups, and one group ends up answering the question in the context of apprehending someone who might commit violence if they remain at large, while another answers in the context of property crime, which would make the discussions incomparable.

I also want to flag that "How often..." (like the very similar "Most of the time..." question we used to have) is likely to generate discussion about experiences, rather than preferences -- A key insight in the last test run was that discussion questions work differently than survey questions

Can we satisfy the desire for ANES overlap by adding some ANES Qs to the pre/post survey, and write our own discussion questions?

RachelAbigail commented 1 year ago

Note: We discussed and agreed on adding the two ANES questions verbatim to the pre and post survey. One of the two (re: use of force) is already covered by a discussion question. The other (re: urban unrest) is not, and we need to add a discussion question to get at the concepts it covers.

Initial suggestion for a discussion question that we can iterate on: What’s the best way to deal with urban unrest? Is it better to use force to quell it or to address social injustice to prevent it?

RachelAbigail commented 1 year ago

GSS Survey Questions re: police

Are there any situations you can imagine in which you would approve of a policeman striking an adult male citizen? (https://gssdataexplorer.norc.org/variables/670/vshow)

Listed below are various areas of government spending. Please indicate whether you would like to see more or less government spending in each area. Remember that if you say "much more," it might require a tax increase to pay for it. C. The police and law enforcement. (https://gssdataexplorer.norc.org/variables/4059/vshow)

ChristopherLucas commented 1 year ago

What’s the best way to deal with urban unrest? Is it better to use force to quell it or to address social injustice to prevent it?

I like this as an alternative. What about the following revision for clarity?

What do you think is the most effective way to handle urban unrest? Should law enforcement authorities use force to put an end to it, or should the underlying social injustices be addressed to stop it from occurring?

RachelAbigail commented 1 year ago

Questions successfully rewritten and deployed in pilot.