Synthesize Federal Header/Footer research & reporting

cindymerrill commented 5 months ago

Description

User story

AS A researcher I WANT to synthesize data from research sessions SO THAT I can summarize findings to share with team and stakeholders

Artifacts

Research document - new paths

Acceptance Criteria - Sprint 100 (5 points)

Preliminary Synthesis -- Phase 1 research

[x] Cleaned up/organized research notes to summarize findings and identify themes
[x] Topline summary of top-of-mind findings written and shared out with team (use this template)
[x] PM and Sitewide UX Lead consulted to see if they have feedback or additional questions for synthesis
[x] Topline summary revised with feedback from Sitewide UX Lead
[x] High priority findings/issues to EnjoyHQ, discuss w/Amanda ASAP to update design

Acceptance Criteria - Sprint 103 (8 points)

Preliminary Synthesis -- Phase 2 research

[x] Cleaned up/organized research notes to summarize findings and identify themes
[x] High priority findings/issues to EnjoyHQ, discuss w/Amanda ASAP to update design
[x] Started writing topline summary

Acceptance Criteria - Sprint 104 (13 points)

Finish topline summary

[x] Topline summary of top-of-mind findings drafted and shared out with team for feedback
[x] PM and Sitewide UX Lead consulted to see if they have feedback or additional questions for synthesis
[x] Topline summary revised with feedback from Sitewide UX Lead

Synthesis and reporting--Phase 1 & 2 together

[x] Recruitment tracker updated with participant demographics
- [x] For no-show participants, data entered as "N"'s
[x] Data synthesis of all findings completed
- [ ] ~~Analyze logo findings from phase 1 separately from phase 2 b/c color changes~~--NO NEED b/c findings aren't different
- [x] Summarize task success data
- [x] Summarize sentiment data
- [x] Summarize "good to know" items from notes, including different ways that AT users navigated using screen readers (e.g., hover by mouse to read, open list of links, line by line)
- [x] Make table of AT software, devices, etc. per participant
- [x] Summarize findings not covered in toplines
[x] Research report in progress

Acceptance Criteria - Sprint 105 (8 points)

Finish Synthesis and reporting--Phase 1 & 2 together

[x] Session recordings/transcripts used to make additional notes and/or pull exact quotes
[x] Research questions from the Research Plan revisited to check whether they've all been answered
[x] Hypotheses from the Research Plan revisited to see which can be confirmed
[x] Research report draft complete

Collaboration with Sitewide UX Lead and others

[x] Research report feedback requested from Sitewide UX Lead and other team members
[x] Sitewide UX Lead requested feedback from VA PO and shared with researcher
[x] All feedback on research report incorporated

Reseach data in EnjoyHQ

[x] Uploaded phase 1 session notes from github into EnjoyHQ (need to copy into new notes in the Data tab)
[x] Uploaded phase 2 session notes from github into EnjoyHQ (need to copy into new notes in the Data tab)
[ ] Downloaded phase 1 edited transcripts from EnjoyHQ -- CAN'T DO easily so skipping
[ ] Downloaded phase 2 edited transcripts from EnjoyHQ -- CAN'T DO easily so skipping

cindymerrill commented 5 months ago

This is the first post-research-session ticket that I'll be working on @jilladams @FranECross

jilladams commented 5 months ago

Acknowledged in planning: after research sessions conclude, Cindy will only have 1 day, maaaaybe 2, of work in sprint. So we're pulling this into Sprint 100 with the understanding that it will start but not finish until Sprint 101.

cindymerrill commented 5 months ago

Also possible this ticket won't finish in the next sprint if I need to start planning Phase 2 of the research so that sessions can start on Feb 1 (which @aklausmeier is targeting). Then I might do a combined report for Phases 1 and 2 of the research later. FYI @jilladams @FranECross

cindymerrill commented 5 months ago

Met with @aklausmeier today and then worked hard to write a long research summary report that's somewhere between a topline and a full report: https://github.com/department-of-veterans-affairs/va.gov-team/blob/master/products/header-footer/federal-standardized-header-footer/research/phase1-no-AT/topline.md. My intent was to write up a summary of what I learned in the research sessions before the holiday break (OOO Dec 25-29) and transition to planning Phase 2 of the research (in January). I don't know when I'll have time to do more data synthesis and write a full report for Phase 1, and it's possible that it will wait until after Phase 2 research is done. FYI @jilladams @FranECross

cindymerrill commented 5 months ago

Added new AC:

Uploaded session notes from github into EnjoyHQ (or skipped if too cumbersome to create new .txt files)

cindymerrill commented 5 months ago

@aklausmeier Feedback on my topline report can be documented here in this ticket, and I can make revisions in January if needed.

aklausmeier commented 5 months ago

@cindymerrill

Thank you for putting together the expanded Topline Report so quickly! Just a few things and mostly for clarity.

1. Usability Successes No.4, "blurry"

Good to note here that page content in this prototype used screenshot images of existing content vs. native in prototype. Phase 2 will use coded page content with updated font family so blurry appearance should not be an issue but usability of font changes is something to to keep an eye on.

2. Authentication Task

Important to clarify if users struggled finding auth menu before AND after sign In as these could have different design implications.
Success rate is the same on desktop and mobile (2 out of 5 had difficulty locating auth menu after sign in. Good to note and interesting.)
Remove design solution in No.6, don't want to lock in on one solution. General note, it's good for a designer to iterate multiple solutions in the creative process w/o having a specific solution in mind since it can unintentionally create a narrow solution focus, this is why I would hesitate to add a prescriptive solution in a research report.

3. Footer

Remove No.12 & No.13 -- both of these call out observations by one participant making it anecdotal and not a pattern or finding

cindymerrill commented 4 months ago

Thank you for your feedback on my topline report, @aklausmeier ! I will work on revisions today, and I have a few comments about your feedback:

Re struggling to find the Auth menu, all 4 users struggled AFTER signing in, not before. I will add this to the report.
I'm reluctant to drop 12 and 13 re the Footer because we may get more feedback on this in Phase 2. I want to track this in Phase 2 to see if any more participants say this--I'll add a caveat to the report.

Has anyone other than you seen this topline report? I'd like Samara and Liz to read the Auth feedback as they're considering revisions to the Auth menu. And Dave C should see the results, too, and Federal CIO should at least see the feedback on the logo. I can show you my revised version as soon as it's ready, and I'm happy to share it more widely then.

cindymerrill commented 4 months ago

@FranECross @jilladams I updated the AC's for sprints 100 and 101, and added point estimates for each. The whole-ticket point estimate is for ALL sprints together (100 + 101 + future), which isn't what's needed for sprint planning.

cindymerrill commented 4 months ago

@aklausmeier Samara just reached out to me to thank me for the great work on the topline, which Dave shared with her in a meeting, so apparently you already shared it out. After I finish making revisions today, I'll let Samara know because she wants to share it with her team.

cindymerrill commented 4 months ago

@aklausmeier "usability of font changes" is not something that we'll get feedback in Phase 2 because most of our participants will be using screen readers. However, seeking feedback on font changes may be another argument for including some low-vision participants using screen magnifiers in Phase 2.

cindymerrill commented 4 months ago

@aklausmeier I've now revised my header/footer topline report to incorporate your feedback. Let me know if you have any more suggestions, or whether I can go ahead and share it with Dave C and Samara.

aklausmeier commented 4 months ago

@cindymerrill The feedback I provided was collective feedback from Dave C. and I (both). We will need 12/13 removed before we share with OFCIO and USWDS. Great idea to keep an eye on those for Phase 2, if it becomes a pattern we can add it into the report. Definitely ok to share with Dave and Samara.

aklausmeier commented 4 months ago

@cindymerrill The current recruitment criteria for phase 2 includes screen readers and screen magnification software so I assumed we might get a mix of both.

However, seeking feedback on font changes may be another argument for including some low-vision participants using screen magnifiers in Phase 2.

cindymerrill commented 4 months ago

@aklausmeier OK, I'll remove 12 and 13 but add them to the Phase 2 convo guide ticket to remind me to track this. Then I'll share in slack with Dave and Samara.

cindymerrill commented 4 months ago

Completed all tasks for Sprint 100

jilladams commented 4 months ago

For sprint 101: focusing on phase 2 research, this piece will become stretch and Cindy will do more on it when phase 2 prep is done. Moving to stretch.

cindymerrill commented 4 months ago

Added a lot more participant quotes about the USA logo to the topline, per Dave's request.

FYI @aklausmeier @jilladams @FranECross

cindymerrill commented 4 months ago

@jilladams FYI I had no time to start this ticket this sprint.

cindymerrill commented 4 months ago

Updated AC's for Sprint 103 FYI @FranECross @aklausmeier @jilladams