Devographics / StateOfCSS-2020

The State of CSS 2020 survey results
https://2020.stateofcss.com
Other
45 stars 14 forks source link

Asking about race/ethnic background/skin tone #5

Closed SachaG closed 3 years ago

SachaG commented 3 years ago

I'm opening this issue to discuss A) whether we should try and ask about respondents' race/ethnic background/skin tone (with the goal being to highlight biases in the industry or in our own methodology) and B) what the best way to do so would be.

SachaG commented 3 years ago

Current idea:

Screen Shot 2020-10-07 at 7 04 50
JoannaFalkowska commented 3 years ago

Not sure how would this emoji list capture e.g. Asian or Native American ethnicities? Ethnicity is not literally a skin colour, and the emojis don't even cover that many of them.

I'd rather follow a list sourced in some government census, e.g. https://www.ethnicity-facts-figures.service.gov.uk/style-guide/ethnic-groups https://www.census.gov/prod/2001pubs/cenbr01-1.pdf

SachaG commented 3 years ago

To explain my thinking on this, I did first look for examples of race or ethnicity questions on governmental censuses and surveys but it quickly became apparent that they were all very specific to the country doing the census, and wouldn't make sense for a worldwide survey.

And while it's true that the emojis don't capture nuances like Asian or Native American, they do capture black/brown vs white, which I think is as good a place to start as any since black/brown groups are the one most often suffering from discrimination (at least in the western countries most of our respondents come from).

So basically, our goal this year is not so much to capture as much data as possible, but more to find a way to "dip our toes" into this domain in a non-intrusive and respectful manner to see what we can get out of it.

JoannaFalkowska commented 3 years ago

And while it's true that the emojis don't capture nuances like Asian or Native American, they do capture black/brown vs white, which I think is as good a place to start as any since black/brown groups are the one most often suffering from discrimination (at least in the western countries most of our respondents come from).

It seems to me that you are trying to be Western-centric and not-Western-centric at the same time, which feels like a bizarre approach.

The entire human experience of ethnicity-based identity and discrimination around the world can't be captured by 5 emojis ranging from "skin-tone-1" to "skin-tone-5"; it is disrespectful to suggest so.

If you want to figure out how many people are discriminated against because of racism, it could be way simpler and more constructive to ask "do you belong to a discriminated or minority ethnic group? yes/no".

Otherwise, capturing people's skin color or ethnicity is, like you pointed out, a wildly complicated topic, and personally, I'd rather have no data than data that's biased towards Western perception of "race".

SachaG commented 3 years ago

The entire human experience of ethnicity-based identity and discrimination around the world can't be captured by 5 emojis ranging from "skin-tone-1" to "skin-tone-5"; it is disrespectful to suggest so.

Yes and we're definitely not trying to suggest that. That's why the question doesn't mention race or ethnicity but is just entitled "Appearance".

If you want to figure out how many people are discriminated against because of racism, it could be way simpler and more constructive to ask "do you belong to a discriminated or minority ethnic group? yes/no".

That's a good point, but I feel like it doesn't let us do a lot to exploit the data. With the emoji approach we could correlate that to e.g. salary in a more visual way.

personally, I'd rather have no data than data that's biased towards Western perception of "race".

That's a good point and I've debated it, but then I'm also worried about hiding behind a goal of perfectionism as a reason for not taking on this question at all, and that also doesn't quite feel right… Did you have specific downsides in mind? What's the worst-case scenario if we went ahead and phrased the question using the emojis?

By the way just because I'm pushing back against your point doesn't mean I don't appreciate your feedback! Thanks a ton for taking the time to contribute to the project!

bogdanbryzh commented 3 years ago

How will be this data manipulated later?

bogdanbryzh commented 3 years ago

Also, I am not sure that such questions will be relevant for Ukraine. There's no discrimination in based on skin tone. At least I haven't noticed anything like. Perhaps because there aren't many blacks or asian here

SachaG commented 3 years ago

We're not sure yet, but we can definitely cross-reference the data by country to help identify local trends.

SachaG commented 3 years ago

Also here's a screenshot of the latest iteration:

Screen Shot 2020-10-11 at 17 42 26
JoannaFalkowska commented 3 years ago

That's a good point and I've debated it, but then I'm also worried about hiding behind a goal of perfectionism as a reason for not taking on this question at all, and that also doesn't quite feel right… Did you have specific downsides in mind? What's the worst-case scenario if we went ahead and phrased the question using the emojis?

  1. People could misinterpret this data as "how much are you discriminated against because of racism on a scale of 1 to 5", while it actually only tells us what's the level of melanin in one's skin. This is Western-centric and misleading.

  2. People who essentially get asked "how brown are you on a scale of 1 to 5" might get offended at a thinly-veiled attempt to capture their ethnicity as a level of brownness. I don't buy your argument about "labeling this section as Appearance" - you don't ask about eye or hair colour, only skin, and there is a reason for that:

with the goal being to highlight biases in the industry or in our own methodology.

  1. The data you'd get would be horribly biased and incomplete. For a simplest example, how do you think these people (all images generated at thispersondoesnotexist.com) should answer? What kind of data will their answers give you?

it's true that the emojis don't capture nuances like Asian

How are East and Southeast Asian people a "nuance" when they could be about 10% of your respondents?


To propose a better solution, IMO the simplest one would be to use SO's list, as they've clearly done their homework already.

SachaG commented 3 years ago

I still have my disagreements with the points you make, but I guess the fact that this emoji scale can engender this kind of debate probably makes it a bad candidate to be used for a survey… The SO question is actually pretty nicely phrased, so I will probably end up just using that. Thanks!

SachaG commented 3 years ago
Screen Shot 2020-10-13 at 20 26 40

The new version of the question. Thanks @JoannaFalkowska !

bogdanbryzh commented 3 years ago

Much better 👍