search

carpentries / assessment

This repository contains assessment data, source code, and analyses for The Carpentries workshops and programmatic assessment.
https://carpentries.github.io/assessment/
Other
5 stars 7 forks source link

Survey demographics skewed towards Engineering/Sciences #20

Open Denubis opened 4 years ago

Denubis commented 4 years ago

I'm working up a clone of the DC survey for our HASS workshop and I realised that the responses skew very heavily towards engineering/science specifics and leave off a generic "humanities" for us HASS folk.

Agricultural or Environmental Sciences
Ancient History
Anthropology
Biomedical or Health Sciences
Chemistry
Civil, Mechanical, Chemical, or Nuclear Engineering
Computer Science or Electrical Engineering
Criminology
Earth Sciences
Economics or Business
Education
English
Genetics, Genomics, or Bioinformatics
Geography and Planning
High Performance Computing
Humanities
Indigenous Studies
International Relations
International Studies
Law
Library and Information Science
Mathematics or Statistics
Media, Music, Communication and Cultural Studies
Medicine
Modern History
Organismal Biology (Ecology, Botany, Zoology, Microbiology, etc.)
Philosophy
Physical Sciences
Planetary Sciences (Geology, Climatology, Oceanography, etc.)
Politics
Psychology or Neuroscience
Security Studies
Social Sciences
Sociology
Space Sciences
Other (please specify)

Is the list unioned with our departments here. Might be worth thinking about for the next round of the survey.

kariljordan commented 4 years ago

Hi @Denubis. Thanks for reaching out about this survey question. We've migrated our surveys over to Typeform, and the question currently reads:

` Please indicate your relevant fields or disciplines.

You can find the survey here: https://carpentries.typeform.com/to/wi32rS

Check out this blog post for more information: https://carpentries.org/blog/2019/10/transition-to-typeform.

We've tried to streamline this question as much as possible given the information we receive in our surveys and through AMY. I'll definitely keep this in mind when we look at the surveys again. Let me know if you'd like to keep this issue open.

Denubis commented 4 years ago

Hi @kariljordan

Yes, I'd very much like to keep this issue open.

Just to put a number on it, first we measure "science fields"

  1. Agricultural or Environmental Sciences
  2. Biomedical or Health Sciences
  3. Chemistry
  4. Earth Sciences
  5. Genetics, Genomics, or Bioinformatics
  6. Life Sciences
  7. Mathematics or Statistics
  8. Medicine
  9. Organismal Biology (Ecology, Botany, Zoology, Microbiology, etc.)
  10. Physical Sciences
  11. Planetary Sciences (Geology, Climatology, Oceanography, etc.)
  12. Psychology or Neuroscience
  13. Space Sciences

Now, "engineeringish"

  1. Civil, Mechanical, Chemical, or Nuclear Engineering
  2. Computer Science or Electrical Engineering
  3. High Performance Computing

Now "Everything else"

  1. Economics or Business
  2. Education
  3. Humanities
  4. Library and Information Science
  5. Social Sciences

13 to 5 is ... not very inclusive, especially for folk already unsure if they're welcome. And the change of granularity between "Organismal Biology" and "Humanities" is quite significant. (Think of it as department versus faculty).

To back up a step, what is the intended purpose of this question?

kariljordan commented 4 years ago

The intended purpose of this question is to understand our audience. The survey was originally developed for Software Carpentry and Data Carpentry. Data Carpentry's lessons are domain specific, hence the science fields. Once we onboarded Library Carpentry as a lesson program, we analysed the data from all of our surveys and we looked at our instructor demographics. We pulled out the most frequent domains. There is an other option for this question as well for people to write in their domain. I summarised the most common write-in responses in the other option, and this is the list we came up with.

I definitely understand your concern and our goal is never to make our learners feel excluded. We use the data from this question in our reports to understand who attends our workshops. I look at this information quarterly and update the survey from there. As we continue to grow and reach new communities we will revise the question.

@maneesha and @fmichonneau will have more information to add as well because they've worked on this question with me. Please chime in!

kariljordan commented 4 years ago

Hi @Denubis, checking to see how we can move forward with this issue. @maneesha @fmichonneau do you have thoughts?

maneesha commented 4 years ago

Thanks for bringing this up.
From the infrastructure/technology perspective, we can ask anything. I think this is more a question of equity and inclusivity - are we treating these fields with equal weight? I think this is a hard question, as we also want to think of it from a practical perspective. I don't think it is practical offer dozens of different choices as that leads to user fatigue.

I agree that terms like "Economics" or "Education" are so broad and many of the ones in the "science" list are much more granular.

Sorry I don't have much more of an answer to this! I just know we need to figure out how to balance what we need to know as The Carpentries program, and what we can do to be inclusive and inviting to our community members.

Denubis commented 4 years ago

Just leaving a note here. Taking a survey from Aarnet, and they asked:

'4. Discipline you most closely relate to; your own words, or use the ANZSRC Classification " (link: https://www.abs.gov.au/Ausstats/abs@.nsf/Latestproducts/4AE1B46AE2048A28CA25741800044242?opendocument). Depending on analysis, this might be a useful pattern. While moving from checkboxes to freetext is always fraught (and I'm not suggesting we move to a more technical solution right now), this feels like the route towards a non-prejudiced solution? (Happy to discuss hinting/autofill, etc if you think that that'd be a useful compliment, but that's out of scope here.)