Watts-Lab / surveys

Library of surveys for deliberation experiments
MIT License
3 stars 4 forks source link

[Add] Panel Demographic Survey #70

Closed JamesPHoughton closed 4 months ago

JamesPHoughton commented 2 years ago

Add a new survey to the repository according to the instructions in the package README.md.

Capture demographic info in the same format as our existing panel data. Items that are relatively static and we don't expect to be influenced by the prior activity.

Survey Title

Static and Semi-static Demographics Survey

Survey Source

Compilation of questions from the CSS lab's existing turk panel Also: https://github.com/Watts-Lab/surveyor/blob/main/surveys/demographics.csv

Survey Overview

<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns="http://www.w3.org/TR/REC-html40">

gender | Participant's Gender. | Options Included: Male \| Female \| Other | What is your gender? | NA -- | -- | -- | -- | -- birth_year | Year of Birth of Participants | Numeric Value | In which year were you born? | NA education_level | Highest Education that participant has received | Options Included: Less than a high school diploma \| High school diploma \| Some college or vocational training \| 2-year college degree \| 4-year college degree \| Post-college degree \| Other | What is the highest level of education you have completed? | NA country | Participant's Country of Residence (typed in Text) | Text String with country name | In which country do you reside? | NA income_min | The lower bound of a selected income bracket range, assumed to be USD, and converted if otherwise listed. | Income Brackets Included: Less than $10,000 \| $10,000-$19,999 \| $20,000-$29,999 \| $30,000-$39,999 \| $40,000-$49,999 \| $50,000-$74,999 \| $75,000-$99,999 \| $100,000-$149,999 \| $150,000 or more | Thinking back over the past year,what was your family's annual income? | Rand et al. 2021 marital_status | Participant's Marital Status | Options Included: Single Never Married \| Married or Domestic Partnership \| Widowed \| Divorced \| Separated | What is your marital status? | Rand et al. 2016 race | Participant's Race. Participants could choose more than one option | Options Included: White & Black or African-American & American Indian or Alaska Native & Asian & Native Hawaiian or other Pacific Islander & Other | Please choose one or more races that you consider yourself to be | NA income_max | The higher bound of a selected income bracket range, assumed to be USD, and converted if otherwise listed. | Income Brackets Included: Less than $10,000 \| $10,000-$19,999 \| $20,000-$29,999 \| $30,000-$39,999 \| $40,000-$49,999 \| $50,000-$74,999 \| $75,000-$99,999 \| $100,000-$149,999 \| $150,000 or more | Thinking back over the past year,what was your family's annual income? | Rand et al. 2021 time_zone | TimeZone of Residence of the Participant | NA | NA | Arechar Rand 2017 DOI:10.1007/s40881-017-0035-119 usual_time | Is Participant doing study at their usual M-Turk Work Time | Recoded to 1-0 | "Do you usually work on HITs at this time of the day"[Yes, No] | NA employment_status | Is participant employed/what is the Participant's employement? | Recoded to the strings | "Are you currently...?" Options were:[Employed for wages, Self-employed, Out of work and looking for work, Out of work but not currently looking for work, A homemaker, A student, Military,Retired, Unable to work] | NA

Notes:

Other questions to add:

Tasks

JamesPHoughton commented 2 years ago

@NettaWeinstein and @tiamoin - these are some of the demographic questions that we have in our existing panel, but the information is incomplete. I'm thinking about building this up as an exit survey to fill in gaps in our panel (and serve as redundancy for our existing data). Would you add anything?

Your original demographic survey was tailored to the UK context, but has some additional questions we don't have here, such as english as a second language, nationality (as opposed to country of residence) and ethnicity.

tiamoin commented 2 years ago

@JamesPHoughton - actually have been looking into this a bit more in recent days. All our guidance is taken from government guidelines and is consistent with national surveys - I updated the nationality questions and disability questions to the following - see below for sources but these are region specific (e.g. presents most commonly answered/recognised options first) so I would follow US guidance for US participants? The wording of questions is based on research on what respondents understand and find easy to answer: How would you describe your national identity? (please choose all that apply) • British • English
• Welsh
• Scottish
• Northern Irish
• Other, please describe

  1. What is your main language? (i: A person’s main language is the language they use most naturally. For example, this could be the language they use at home.) [ ] English [ ] Other, write in (including British Sign Language)

  2. How well can you speak English? [ ] Very well [ ] Well [ ] Not well [ ] Not at all

  3. Do you have any health conditions or illnesses which affect you in any of the following areas? Note: In answering this question, you should consider whether you are affected in any of these areas while receiving any treatment or medication or using devices to help you such as a hearing aid for example. 1 Vision (for example blindness or partial sight) 2 Hearing (for example deafness or partial hearing) 3 Mobility (for example walking short distances or climbing stairs) 4 Dexterity (for example lifting and carrying objects, using a keyboard) 5 Learning or understanding or concentrating 6 Memory 7 Mental health 8 Stamina or breathing or fatigue 9 Socially or behaviourally (for example associated with autism spectrum disorder (ASD) which includes Asperger’s, or attention deficit hyperactivity disorder (ADHD)) 10 Other (please specify) 11 None of the above

(Harmonised Questions and Responses for England (online) – Nationality https://analysisfunction.civilservice.gov.uk/policy-store/national-identity-harmonised-standard/ (Harmonised Questions and Responses for England (online) – Ethnicity: https://analysisfunction.civilservice.gov.uk/policy-store/ethnicity-harmonised-standard/#paper-guidance) (Harmonised Questions and Responses for England (online) – Impairment: https://analysisfunction.civilservice.gov.uk/policy-store/impairment/) (ONS: National identity, ethnic group, language and religion question development for Census 2021) https://www.ons.gov.uk/census/censustransformationprogramme/questiondevelopment/nationalidentityethnicgrouplanguageandreligionquestiondevelopmentforcensus2021#research-that-led-to-the-recommended-census-2021-question-designs

tiamoin commented 2 years ago

@JamesPHoughton in the U.K. they guide us to use the word "ethnicity" rather than "race" but I think there are cultural differences in this term between the US and UK

JamesPHoughton commented 2 years ago

Interesting! Sounds like we may want to structure a demographics survey that adjusts to context - if the country of residence is UK vs US vs elsewhere. Should be doable. =)

JamesPHoughton commented 2 years ago

notes from discussion with @tiamoin @nettaweinstein

xehu commented 2 years ago

Random note: but can you change "is English your first language" to "is English your native language?" Because English is my second language but I speak it natively (due to learning it at age 5). I always struggle with those questions because I feel like I can't accurately answer them.

markwhiting commented 2 years ago

Are you thinking that some of these questions would depend on country? Might it be better to have those as a separate survey (it probably would be in the surveyor system but perhaps less so here)

markwhiting commented 2 years ago

Are you thinking that some of these questions would depend on country? Might it be better to have those as a separate survey (it probably would be in the surveyor system but perhaps less so here)

TutiGomoka commented 2 years ago

Also need to validate the country column so they can only enter valid text. Not emojis or something like that

xehu commented 2 years ago

Why are income min and income max included separately? I am a bit confused with how it works.

TutiGomoka commented 2 years ago

So: @JamesPHoughton I think the way in which the variables are listed in the codebook isn't the same way they are asked. E.g., we derive income min and income max from the one income question we ask

TutiGomoka commented 2 years ago

We also haven't been asking usual time I think that was a variable in the previous study that Abdullah sent us

markwhiting commented 2 years ago

I think this is done, so hiding it from view on https://github.com/orgs/Watts-Lab/projects/8

JamesPHoughton commented 2 years ago

Should split off "do you normally do hits this time of the day" into a separate turk habits survey

JamesPHoughton commented 4 months ago

We have two demographics surveys now, a short and a long. We can continue to refine them as we go, i don't see it being a problem to modify the survey as these are not "reactive" (ie, demand effects) in the way that some others might be.