Team Review of Terminology Survey (Google Form)

kelseylondon commented 1 year ago

Overview

We have received approval to carry out the Terminology Survey, originally developed in Summer/Fall 2022. To (re-)familiarize ourselves with plans for this survey, we need all team members to (1) review the survey and (2) provide feedback and suggestions.

Action Items

[ ] Review the Terminology Survey.
[ ] Please provide feedback and suggestions on the research plan as a comment to this issue.
[ ] Once you have completed the above steps, please check off your name below:
- [x] Yumeng
- [ ] Kelsey
- [x] Sam
- [x] Kevin
- [x] Sofia
- [ ] Analicia
- [ ] Travis

Resources/Instructions

Related issues

729
714
708
669
652
603

sofialaguna commented 1 year ago

Suggestions for survey:

For question 1, "Please check which option applies" - I suggest reformatting the question as a screening question by adding a response option that states "I am not actively clearing my record, helping someone, or have cleared my record in the past". If respondents choose this option, we can lead them to submit the survey without filling out the other questions. This will help us weed out responses that are not applicable for our study.
For question 2, add "to clear your record" at the end of the sentence: "In which state and county are you applying (or have applied)"
I suggest adding question 7 ("Did you use this type of document...") before question 5 ("If you are familiar with this kind of document"), but keeping the description from question 5 ("In this process, some applicants are required to write a document...") before asking question 7 to provide context on the document. I think that Question 7 can be used as a screening question that asks participants if they are familiar with the personal story document (or whether it was required to complete).
- If participants respond yes they are familiar/it was required, then they can answer question 5 that asks about terms they have used in the process.
- If participants respond no they are not familiar, then they can be redirected to finishing the survey.

I hope my suggestions make sense, but if not I am happy to explain in more detail. I have used screening questions on google forms before and I find them helpful for redirecting respondents from answering complete surveys if they don't match the criteria we are looking for.

kevinmoutoucarpin commented 1 year ago

Suggestions for the Terminology Survey :

For question 1 (initial version / IV), there are 2 things :
- I would replace “I am actively helping a friend or family member […]” by “I am actively helping someone (friend, family member, etc.) […]” because maybe they’re helping someone other than a friend or a family member and we may not want to exclude them
- I would add another option : “I have already helped someone (friend, family member, etc.) to clear their criminal record” because we don’t have the option of having done it in the past for someone else
For question 3 (initial version - IV), I would split it into 2 questions. The logic behind it is to avoid double-barreled questions (i.e. a question that essentially includes more than one topic and is asking about two different issues, while only allowing a single answer), which is the case here because we ask if they used or encountered these terms and at the same time we ask them about their frequency. For that :
- First, we ask the question “In your experience, which terms have you used referring to the process of clearing a criminal record?”. The differences here are that we replace the frequency scale by "Yes/No" (one column for each) and we remove “something else” in the options
  - If they answer “No” to all of them, they can go to the question 4 (initial version - IV) if there are other terms they used that are not mentioned in the options
- Then, the 2nd question of question 3 (initial version - IV) would be : “At what frequency, if at all, have you used referring to the process of clearing a criminal record ?”.
  - Replace "How frequent" by "At what frequency" due to the presence of an assumptive leading question (i.e. "How frequent"), which can guide the participant into answering in a certain way.
  - Remove the “something else” in the options
  - Replace the current scale with a 5-point scale like this one : "1 - Never, 2 - Rarely, 3 - Sometimes, 4 - Often, 5 - Always" (source)
Just before question 4 (initial version / IV), I would add a question like this : “On a 5-point scale from 1 (Poor) to 5 (Excellent), how do you rate the following terms as a way to refer them as the process of clearing a criminal record" :
- Options possibles :
  - Record Clearance
  - Expungement
  - Record Sealing
- Use a 5-point scale like this one : "1 - Poor, 2 - Fair, 3 - Good, 4 - Very Good, 5 - Excellent" (source)
For question 4 (initial version - IV), I would rephrase “If there are other terms you have used that didn’t appear in the list previously, please specify your answer for each term, the appropriate frequency, and rate them based on a 5-point scale from 1 (Poor) to 5 (Excellent) like in the previous question. If you did not, you may skip this question.”
For question 5 (initial version - IV), I would split into 3 questions. The logic behind it is to avoid assumptive leading questions (i.e. questions that make assumptions about how others feel, what they’ve done or will do). For that :
- First, I would keep the 2 first sentences and cut “If you are familiar with this kind of document, what do you usually call it? Which term(s) have you used or encountered?” and replace it with “Are you familiar with this kind of document?”. And I would replace the frequency scale by “Yes/No” (one column for each) and remove “something else” in the options”.
  - If they answer no, they can move on to the question 7 (initial version / IV) “Did you use this type of document […] when applying for record clearance?”
- Then, the 2nd question of question 5 (initial version - IV) would be : "Which term(s) have you used to call this kind of document?" with a "Yes/No" (one column for each) and remove "something else" in the options :
  - A personal letter
  - A personal statement
  - A declaration letter
  - A petition letter
  - A support letter
- The 3rd question of question 5 (initial version - IV) would be : “At what frequency, if at all, have you used these terms for this kind of document?” (remove “something else” in the options and replace the current scale with a 5-point scale like this one : "1 - Never, 2 - Rarely, 3 - Sometimes, 4 - Often, 5 - Always" [source]) :
  - A personal letter
  - A personal statement
  - A declaration letter
  - A petition letter
  - A support letter
Just before question 6 (initial version / IV), I would add a question like this : “On a 5-point scale from 1 (Poor) to 5 (Excellent), how do you rate the following terms as a way to call this kind of document"
- Options possibles :
  - A personal letter
  - A personal statement
  - A declaration letter
  - A petition letter
  - A support letter
- Use a 5-point scale like this one : "1 - Poor, 2 - Fair, 3 - Good, 4 - Very Good, 5 - Excellent" (source)
For question 6 (initial version - IV), I would rephrase by “If there are other terms you have used that didn’t appear in the list previously, please specify your answer for each term and the appropriate frequency and rate them based on a 5-point scale from 1 (Poor) to 5 (Excellent) like in the previous question. If you did not, you may skip this question. If you did not, you may skip this question.”
For question 7 (initial version - IV), I would remove “or are you planning to use it” because with this, it implies that we are using a future-based question and we are trying to predict their future behavior and people are bad at predicting their future behavior (see my comment in the EA Research Plan - Terminology Survey).

Sorry I know it's a mess and really hard to read/digest so to make it easy for you, I put it in a PDF doc here so that you can see how it looks like : EA Teminology Survey (Kevin's suggestions).pdf

tntalcott commented 1 year ago

Q1:
- I would suggest adding an option for “I work with individuals who are attempting to clear their criminal record in my profession (e.g., lawyer).” ? Or, is this the wrong survey for that?
- This question could be used as a screening question, if none apply then there is no need for them to fill the rest of the survey (as others have already suggested)
Q2:
- Maybe ask the respondent to list the county then state? This would help to keep it in line with traditional naming, but it would make sorting more difficult.
- Would it be worth asking if they’ve attempted this process in more than one county, and, if yes, ask if they noticed differences in naming between the counties?
Q3:
- Since we provide the terms for them, it might be better to word the question as “how often have you or used or encountered the following terms in relation to the process of clearing a criminal record?”
Q4:
- Maybe instead of including the “something else” option in Q3, you could ask here “Are there any other terms that you have commonly used or encountered in relation to the process of clearing a criminal record that was not included in the prior list? If so, then please specify. Otherwise, you may skip this question.” (Looks like Kevin had a similar idea)
Q5:
- This question seems to be trying to do too much at once. I would recommend breaking it up into multiple parts. The first part should ask if they are familiar with this kind of document, and the second part should ask how familiar they are with each term.
Q6:
- As with the previous instance of this type of question (Q4), I would recommend changing it to “Are there any other terms that you have commonly used or encountered in relation to this kind of document? If so, then please specify.”
Q7:
- It seems suboptimal to have this question here, this should be asked before Q5.
Q8:
- The survey doesn’t indicate what “our free tool” is, and the way it’s worded may be interpreted as the tool is currently ready for use. I would suggest rewording it to say something like “…or are interested in using our free tool to assist in writing these documents (currently under development),…”
New Q9?:
- If they answer yes to question 7, would it be worth adding a question that asks the respondent “how likely would you be to seek out and use an online writing assist tool to help write the document?”

hackforla / expunge-assist