Closed dialectic51 closed 1 month ago
NOTE: Compare this categorization of names with the categorization already conducted in the previous thematic analysis of RP001, which can be found in the google drive.
Here are potential domain names and categorization for the interview questions for Internship RP01. They can be used once the lead analyst creates a summary template for RQA (see the checklist created for this issue). The domain names and categorization are based upon the advice provided by Linda in her presentation on RQA.
Role and responsibilities
Learned skills and discovery
Problem-solving
Engagement and accountability
Social networking
Pain points
Choice of HfLA
Prior expectations
Recommend, endorse, volunteer
Don’t require qualitative analysis
We need to conduct Rapid Qualitative Analysis (RQA) of interview transcripts or other qualitative materials for RP[Replace 000].
[ ] 1.) Develop a Research Plan
[ ] 2.) Collect data/conduct interviews
[ ] 3.) Transcribe and read interviews
[ ] 4.) Create a summary template for use by the team in a google doc
[ ] 5.) Take summary template for a "test drive" and assess its usability and relevance
[ ] 6.) Divide transcripts across team and summarize (can divide by site, role, etc.)
[ ] Track who has which transcript to review
[ ] Post completed summaries to appropriate spot on shared google drive
[ ] 7.) Transfer summaries into a matrix (respondent x domain)
[ ] 8.) Conduct matrix analysis
[ ] 9.) Identify themes within domains
[ ] 10.) Prepare report/presentation
@bradyse I wrapped up issue #330 and created this (issue #338) in order to put together the checklist for conducting rapid qualitative analysis. I have completed the checklist in this comment in this issue.
When you have time, could you take a look and make sure I'm doing what we discussed at last week's meeting? If it looks like I'm on the right track, then I will contact Linda, have her review the checklist, and then make changes based on her feedback. We can also discuss this at our meeting tomorrow.
@dialectic51 This looks good. Go ahead and send the comment link to Linda. Ask her to quickly look through it and determine if the basic steps are there. If there are any glaring errors, she can point those out. Otherwise, I wouldn't expect her to spend more than 20 minutes or so reading through the whole thing.
@lkawentel I have completed a checklist for conducting Rapid Qualitative Analysis (RQA), which I have placed in the comment above. I have based the checklist on your answers to Sara's questions, your presentation, and the materials your presentation links to.
When you have time, please review the checklist and identify any errors or things that need to be added to the list. Sara says this shouldn't take more than 20 minutes or so.
(I have also put together a list of potential domains based upon your presentation. I'm going to continue to revise them as I gain more expertise in RQA, but feel free to look them over if you have the time).
@bradyse
I have put in a request (via github and slack) for Linda to review the checklist.
I also made a few revisions to the checklist to address the points you raised at yesterday's meeting. This includes:
Update on RQA for RP001: I have created templates for both the Summary Template and the Matrix Analysis to be used once we start conducting the RQA (located in this folder of the Internship google drive).
They are based off of slide #23 and slide #33 of Hamilton’s presentation and used in steps 4 and 7 in the RQA checklist. I have placed links to the templates in the instructions/resources section of issue#338. (Note: these are templates, but I haven’t been able to figure out how to turn them into actual templates in google docs and sheets).
The summary template was pretty straightforward from Hamilton’s slides. However, I made some changes to the Matrix in order to bring in two additional columns next to each domain: 1.) analytical memos and 2.) themes and trends. (Constructing analytical memos and identifying themes and trends are key steps in conducting a matrix analysis).
I have also placed in bullet points in each cell under the domain columns. The bullet points in the Matrix should be taken directly from the bullet points that will be created in the Summary Templates by a coding team. However, bullet points are a little wonky in Google Sheets, so I have a link to directions on how to create them at the bottom of the document.
(This note was also sent via Slack to Sara, Linda, and Angela).
@dialectic51 Hi Brian! Welcome back from the break. We are confirming everyone's interest and availability. Please provide a new update to this issue you have been working on. Thank you.
Please provide update:
@pandanista I hope you had a good break!
@Dewey-Tran To prepare you for reviewing the checklist, look over Linda's Presentation to UXR Team on How to Conduct Rapid Qualitative Analysis and the Transcript of Linda's Presentation on RQA, which are also located under the Information/Resources section in the Overview. These are the documents that I based my checklist on.
I had reviewed the list @dialectic51 created, here are my suggestions of tasks to include.
1.) Develop a Research Plan
2.) Collect data/conduct interviews
8.) Conduct matrix analysis
@adarosh I am assigning you to review the checklist above. Let me know if you have any questions. Thank you!
Hi @dialectic51 and @pandanista - please see below for feedback. Hope some of these suggestions help- if you have any questions about my comments, just let me know. Looks great so far!
1. Develop a research plan:
2. Collect data/conduct interviews:
3. Transcribe and read interviews OR listen to recordings and take notes:
4. Create a summary template for use by team in word document:
5. Take summary template for “test drive”:
6. Divide transcripts:
7. Transfer summaries:
8. Conduct matrix analysis:
9. Identify themes within domains:
10. Prepare report/presentation:
Currently updating RQA Checklist based on the feedback provided by Angela. I have completed incorporating the suggestions for Step 1 (Develop a research plan). I should be able to finish incorporating the rest of the feedback either later today or tomorrow.
Finished incorporating the feedback provided by Angela into the RQA checklist. The RQA Checklist should be ready to be taken through an initial run to see how well it works, making additions or changes based on the experience.
I will remove myself from the issue at our next meeting so that I can work on more time sensitive materials, such as RP020.
Need to assign a team member to revise the checklist above.
Here is an example from RP017's checklist.
@pandanista I reviewed all the comments and related issues. Before I start revising the checklist, I think I need to know more about the context and objective of this task. We can discuss this in today's UX meeting.
A minor issue is that I do not have access to the following folders/docs:
Hi @jinyan0425, I just found the instructions to be added back to the UI/UX COP Google Drive. See detailed info here.
Hi @jinyan0425, I just found the instructions to be added back to the UI/UX COP Google Drive. See detailed info here.
@pandanista Thanks, Ying! Just filled out the form.
@pandanista
I read through the transcript, which is not the transcript of Linda's presentation but that of Hamilton's presentation back in 2013. So, the label of this resource is incorrect: Linda's Presentation to UXR Team on How to Conduct Rapid Qualitative Analysis. We may need to correct it. On the other hand, it will be very useful if we can find the actual transcript for Linda's presentation.
I would like to confirm we will fully follow Hamilton's Process/Approach, which relies on verbatim transcript instead of identifying themes directly from audio recordings. Hamilton's Process/Approach is an early approach to RQA. See the following expert from a recent method paper on RQA -- Nevedal et al. 2021. This paper was also cited in Linda's presentation on pp.10:
Hamilton developed a rapid qualitative analysis approach that summarizes transcript data into templates using domains aligned with interview questions; summary points are then distilled into a matrix organized by domain and participant for analysis and interpretation [18].
The rapid approaches described by Hamilton and Gale et al. rely on verbatim transcripts, which means teams must wait for transcription to be completed to proceed with rapid or traditional analyses. In contrast, Neal et al. [28] developed an approach to rapidly identify themes directly from audio recordings.
@pandanista This is tentative and developing. I made some comments about the draft at the end.
1. Get Familiar with the Hamilton's Process of RQA
2. Create the Transcript Summary Template
3. Conduct Transcript Summaries
4. Conduct Matrix Analysis
5. Prepare reports/presentations
COMMENTS: 1) Many researchers may not be familiar with RQA, especially Hamilton's Process. Therefore, I think the Step should be to become familiar with this method. In Step 1, I do NOT want to mention Nevedal et al., 2021, which could potentially confuse researchers if we exclusively focus on Hamilton's Process (see my comment above). 2) Steps 2 and 3 can be combined as Conduct Transcript Summaries. 3) The sub-points in each step may still be too detailed. If so, we can remove some of them. 4) On the other hand, RQA is quite complex, with many nuances even at each lowest-level step in the checklist. For instance, there are detailed instructions on how to "Compare the summarizing 'styles' across the team". If we remove this bullet point from the checklist, we may still need to reintroduce all these details as action items in the actual data analysis phase/issue (e.g., #493) 5) Another challenge is the high flexibility of RQA, where many steps need to be customized for specific RPs. This requires careful construction of the actual data analysis action items tailored to each RP.
@pandanista
I read through the transcript, which is not the transcript of Linda's presentation but that of Hamilton's presentation back in 2013. So, the label of this resource is incorrect: Linda's Presentation to UXR Team on How to Conduct Rapid Qualitative Analysis. We may need to correct it. On the other hand, it will be very useful if we can find the actual transcript for Linda's presentation.
- I don't think we have the actual transcript from Linda's presentation because it was not recorded.
- Resolved.
I would like to confirm we will fully follow Hamilton's Process/Approach, which relies on verbatim transcript instead of identifying themes directly from audio recordings. Hamilton's Process/Approach is an early approach to RQA. See the following expert from a recent method paper on RQA -- Nevedal et al. 2021. This paper was also cited in Linda's presentation on pp.10:
Hamilton developed a rapid qualitative analysis approach that summarizes transcript data into templates using domains aligned with interview questions; summary points are then distilled into a matrix organized by domain and participant for analysis and interpretation [18].
The rapid approaches described by Hamilton and Gale et al. rely on verbatim transcripts, which means teams must wait for transcription to be completed to proceed with rapid or traditional analyses. In contrast, Neal et al. [28] developed an approach to rapidly identify themes directly from audio recordings.
- Since our transcripts are mostly ready, I think we could follow Hamilton's Process/Approach.
COMMENTS regarding RQA's updated checklist:
@sunannie27
Please review the checklist. Let me know if you have any questions.
If the checklist looks good, the research leads will use that to make RP001's data analysis new issue.
(not transcripts_) Is this a typo? Are you trying to italicize as well as bold?
(not transcripts_) Is this a typo? Are you trying to italicize as well as bold?
YES, KC! Thanks for catching it. Edited.
@jinyan0425 @pandanista Checklist looks good. Will kickback to research leads.
Dependency
324 transcription needs to be de-identified
Overview
We need to develop a step-by-step checklist for conducting Rapid Qualitative Analysis (RQA) at HackforLA. This checklist will be used to reanalyze data from the RP001: IS21 Internship Exit Interview. This checklist can also be used as the basis for developing a Template for conducting RQA and the starting point for creating a Guide.
Action Items
lead analyst (Angela D.)Dewey Tran for reviewResources/Instructions
330