Mixed methods to qualitative insights from research

[MasSamH]

Overview

We will enrich and validate research findings with any qualitative research that emerges from quant based answers (1-5 scored questions)

Action Items

• [x] Otter AI sign up
• [x] Work flow
  • [x] Otter AI create transcript from voice recordings
  • [x] Create second google form template to capture those additional transcribed findings / import answers into
  • [x] Transcription puts into buckets of what was said where
  • [x] Validate the transcript template after a few interviews
  • [x] Field about who they are - use UXR number + notes field
  • [x] Transcription testing
    • [x] Manual setup of spreadsheet with defined fields per comment
    • [x] Scripted Output from Transcriptions (Python) [details to be found in #49]
    • [x] Testing Procedure UXR [details to be found in #59]
• [x] Test methodology between Project Team / Jonathan to test [details to be found in #59]
• [x] Test one UXR manually with pilot [details to be found in #59]
• [x] Rollout training to wider UXR group [details to be found in #59]

Notes:

• People who signed up to do UXR research is a role
• People signing up for transcription is a different role
• We want to have a validation step, therefore interviewer should not be the transcriber

Resources/Instructions

Transcription Clean Up Template Transcription Sheet Template Research Analysis Plan

[MasSamH]

Transcription Clean-Up Going to have 1 sheet per interview Vertical not horizontal spreadsheet

Manual Spreadsheet - Thad @thadk Field allocations: First column field name on left Second column Existing text (form submission) Third column exactly what OtterAI says (taken from audio) Fourth column actual clean input Fifth column is 'notes'

Scripted Output from Transcriptions - Jonathan #49 Generate script (Python) that wrangles which text belongs to which field in the survey - based on 12 interviews Python script will download results from google form all transcripts from OtterAI

• Match which text matches which field in the survey
• Output goes into one spreadsheet

Testing Procedure UXR - Marianne @MarianneAnthonette - #59

• Pilot
• Establish what difficulties there could be
• Revise training documentation
• Expand pilot to UXR (Tan)

[thadk]

I created a sheet in the Templates folder that can get us started on this. [copied to #62]

Procedure for use:

1. Please do not share it as-is: it has full access to our interviewees results.
2. Pick which row of the main results you want in the first orange part of the second sheet. (I didn’t come up with any fancy ID scheme: maybe something like the one we made for audio upload: 003_codeforcharlottesville_2020-10-22_Jonathan-Kropko, but there are a few more steps to do this)
3. I have re-ordered the fields only in the first sheet/tab.
4. Next select-all and copy and paste both the values (this way you don’t get the formulas) and the format from the first tab of this document into a totally new spreadsheet in a transcriptions folder. Naming scheme maybe like the proposed ID or OtterAI title?
5. In the new spreadsheet, I recommend freezing the first header row in the View->Freeze->1 row.
6. Share the new spreadsheet with the transcriber and make sure they have access to the corresponding otter.ai.

[MasSamH]

Transcription Template

[ExperimentsInHonesty]

@MarianneAnthonette will add columns to the UXR schedule spreadsheet with "Recording Uploaded (TRUE)" and "Transcription Done By (Persons name-wip/ persons name)"

[ExperimentsInHonesty]

We have created a folder called Transcription Storage where we are storing: @MarianneAnthonette @thadk will decide what is going into the folder and organize accordingly so that it is easy for UXRs who are going to do the work of matching up the words with the answers into the individual spreadsheet templates.

[ExperimentsInHonesty]

@MarianneAnthonette will get started.

[ExperimentsInHonesty]

@thadk will move rows on template to match the interview question order.

[thadk]

Mari's notes while doing the transcription + action items

• [x] Copied the sheet from template to new spreadsheet. Deleted the current values (needs formula), then recopied the values into the new spreadsheet.
  • [x] Edit transcription template procedure + document it in the template.
• [x] Copied audio name. Do we want the naming scheme to include UXR name at all?
  • [x] Don't include the UXR name when naming transcription sheets.
• [x] Create a document that outlines Interview Transcription Procedure in the Transcription Storage folder. Has transcription tracking link.
• [x] Spreadsheet should have specific link to Otter.ai audio for the transcriber's reference (keeps every asset transcriber needs together)
  • [x] Insert Otter.ai audio link for each transcription sheet and allow access to Otter.ai link to the specified transcriber.
• [x] Exported text(?) How are we handling the Otter.ai exact text for each sheet
  • [x] Follow-up with Jon
• [x] Tag people on Otter.ai beforehand
• [x] Make a new sheet in UXR Schedule or a separate sheet to keep track of transcriptions (maybe move Interview Response List Sheet?)
• [x] First pass (Otter.ai bucketing) took ~ an hour. Making the adjustments took a bit less than an hour
  • [x] Include guesstimate of transcription completion in procedure document.
• [x] Include how to use notes column in procedure document.
• [x] Need for verifying the rankings before even doing quant analysis
  • [x] Include in procedure document.
• [x] Definitely should start on creating a research analysis plan.
  • [x] Begin drafting a research analysis plan.
• [x] Adding things to the assets sheet: What would be the absolute minimum needed so that it can make it on the sheet
  • [x] Create an internal assets sheet of assets that aren't well documented (?)

[ExperimentsInHonesty]

@MarianneAnthonette will go back to the spreadsheets and delete the row 158 artifact

[ExperimentsInHonesty]

Notes filed could be used to identify problems with the audio in case we need to follow up on what they said.

[ExperimentsInHonesty]

@thadk added the urx of the transcriber

[MasSamH]

@MarianneAnthonette @thadk

• [x] Need to determine best place for tracking progress of transcriptions complete
• [x] Need url from each of the Otterai transcribe interview and pull that into the primary sheet [Brigade Contact Sheet]
• [x] Create new location for spreadsheet tracking progress

Separate points Marianne is investigating:

• [x] How are we capturing processes outside of what are documented
• [x] Marianne starting on outline for research plan.

[MarianneAnthonette]

Additional fields to add to transcription template:

• [x] Length of interview
• [x] Otter.ai link
• [x] Transcription status

[MarianneAnthonette]

@MasSamH Typed a rough outline of the research analysis plan. Definitely not set in stone, nor fleshed out, but can possibly inform future issues to make.

[MasSamH]

@MarianneAnthonette @jkropko @raquelnish

• Review and straw man of Research Analysis plan by Raquel and Jonathan
• Create issues matching elements in the Research Analysis plan and add to OKR document - with Sam
• Marianne reaching out to Research Analysts volunteers - capturing volunteers in UXR availability form and add columns to capture which areas they are interested in.
• Determining how we want to code the qual data - consult with Thad

Resource: research analysis plan

[MarianneAnthonette]

• @MasSamH Made more progress to the Research Analysis Plan. The text in bold can be made into issues (with the other bullet points made into action items). Probably skip making issues for the 'Analysis' section for now.
• Good idea to set up a discussion with @thadk, @jkropko, and @raquelnish (and other research analysis volunteers with experience: Tan, Helen, Reem, Myranda) at the end of next week or the beginning of the following week about fleshing out the research analysis protocol (what we want to get out of the research data). --> I can send this Lettuce out.
• Contacted people interested in research analysis and waiting for their responses

[MasSamH]

@MarianneAnthonette Discussion to be scheduled after 25th Nov. to Jon, Thad, Raquel to complete, plus to other volunteers.

[MasSamH]

@MasSamH Create issues matching elements in the Research Analysis plan and add to OKR document - with Sam

[MarianneAnthonette]

Sam will work on this and provide an update on Saturday's meeting.

[MasSamH]

@Baleja We will cover the Research Analysis plan related issues as part of the OKR/WBS audit that Serena is leading.

[Baleja]

@MarianneAnthonette Next step is for Bonnie, Serena, celeste & Mari to meet and go through this board

[Baleja]

Will ask for time after thursday meeting.

[celeste-hub]

We plan to meet on Saturday 19/12.

[MasSamH]

@MarianneAnthonette

• One more clean up session between Marianne, Bonnie, Raquel - meeting on 29th Dec.
• Issues that need focus: 55, 57/59/65