UX | AuthMenu Claims and Disability Rating | Research Plan

[mtcA6]

Background

We need to create a research plan for the Auth Menu Expansion we're testing in the new header/footer

Tasks

• [x] Write research plan
• [x] Review/share with PM & PO

Acceptance Criteria

• [ ] Plan is ready to submit and kick off the research

[andaleliz]

@mtcA6 sharing about a loose timeline to help with convos/planning while I'm out. This assumes that:

• the sitewide team's 1/17 target of having the prototype completed doesn't shift
• the sitewide team will make good progress while I'm out and we'll have a strong sense of what their prototype will include as we get into the research plan and convo guide work.
• This work will be my focus in the sprint when I come back

Jan 9-12: Work on the plan and conversation guide Jan 16: kick off recruiting Jan 29-Feb 1: Conduct sessions Feb 2: Topline summary completed Feb 5-9: Synthesis in progress Feb 12 (or thereabouts): Read out to team

[mtcA6]

I think they're on track for 1/17, they do not intend to add the two additional items to the auth menu so we'll need to do that. See the comments on #16573

My only question is, will you need Adam's help to add those to the codepen?

[andaleliz]

@mtcA6 I won't need Adam's help to add those to the CodePen, but I might need his help to build out some other pages. I will know more once I start working through the research plan (so in about 24 hrs).

[andaleliz]

@Samara-Strauss and @mtcA6 I've made a lot of progress on the research docs for this work and am ready for your input. Here's some more context about what I'm thinking. I have some questions in bold that I'd love your feedback on.

Research plan

• I’d like to start the session with a really simple exercise of asking the participant to imagine their ideal “user menu”. This will be a good way to understand their expectations of a menu like that, without priming them by showing them what we have designed/prepared.
• I’m planning on updating the user name in the prototype to the participants name before each session to reduce the amount of imagining a person has to do. I think this will be fairly fast and easy.
• The recruitment criteria for screen readers on mobile is because I’m particularly interested to know their experience with the auth menu when it’s nested under a generic “menu” button instead of their name being read aloud in the header like it is desktop.
• The recruitment criteria around health care is because I want to avoid falsely inflating the results to prioritize health care

The research plan is ready for your review.

Conversation guide

This is a bit harder to make progress on without the full prototype, but I've seen enough bits and pieces to make solid progress here. Here's a draft of the guide you can review. It feels light, but I expect to add more guidance prompts once I have that. The core questions are there though. A few notes:

• I'm going to start the person on an unauth page from the sitewide prototype (I think it's something about life insurance). For our purposes, I don't really think the starting point matters and this saves us the step of building a page. They'll sign in on that page, and be redirected to an auth version of the page, which mirrors the prod UX.
• I picked find "Profile" and find "View Disability Rating" as the tasks. Profile is a fairly obvious thing to have under a user menu so this felt like a good starting point to see if they can use the auth menu. Disability rating felt good as an alternative to test how intuitive it is for that to be a link in a user menu, and whether not people can recall how to use/find the auth menu. Do you think a 3rd task is needed? If so, we could send them to My VA.
• For the prototype, I'm going to see how easy it is to copy/paste from prod into the protoype and make a decent page. If it's at all difficult, I think a very simplified version of these pages will work. For example, an h1 that matches the link text they clicked from, and content that says something like "thank you. please wait for the moderator before continuing". Since we're not testing content on these pages, I don't think we need to build them out to be realistic. Do you agree with this approach?

I'm excited to hear your feedback!

[Samara-Strauss]

Hey Liz -- acknowledging receipt, and that I will review these tomorrow.

[andaleliz]

Hi @Samara-Strauss, thanks for your feedback on our call today and all the great points you made! I updated both documents in the following ways:

1. Deprioritized the co-design exercise. As part of the post-task interview, I will ask questions to dig into expectations about what's in the menu, the ordering of the links, and what might be missing. For sighted participants, I'd still like to use the mural board as a visual aid to guide that part of the convo, and reflect any updates they want to make. I think it'll be an interesting experiment in getting Veterans more involved in the process, and could make for a richer conversation. I don't think it'll add any more time to the session.
2. Updated tasks so they are as follows:
   • Task 1: navigation to view disability ratings
   • Task 2: navigation to letters
   • Task 3: navigation to profile (updated task to update phone number since we did address in previous research)
   • Task 4: optional if needed: review auth menu. It occurred to me that there's a chance someone will completely fail and not ever find the auth menu. Unlikely but want to be prepared

Both docs are available for review in this folder. Let me know if this feels better to you. Thanks again for your comments!

[Samara-Strauss]

@andaleliz this is MUCH better. Thank you! Looks good to go for a review from Amanda or Cindy.

[andaleliz]

Thanks @Samara-Strauss! I asked them in Slack to drop any feedback here in the next day or so. I'm also checking with Adam to try to get an ETA for when we'll have the start of a prototype, so we can drop a link in the plan and kick off recruitment. We have an awesome start with what the Sitewide team has provided us.

[aklausmeier]

@andaleliz A few thoughts that came up while I was reviewing your research plan.

• We typically kick off research with Research Ops pending finalized research plan, convo guide and prototype, curious if you are planning to kick off prior to your team's revised prototype being ready for Shane's review?
• Feel free to pull OCTO and OFCIO research priorities from our research plan and anything else you might still need. Might be good to link back to our research plan as your research is fulfilling "Phase 3" that we reference in our research plan.
• Cindy spun up a CAIA accessibility review ticket for research plan, convo guide and prototype review, linking here for reference so you have context of concerns they brought up
  • including screen reader sessions being 2X in session length
  • accessibility specialist scheduled/present for each SR session
• In our research we discovered that folks voiced confusion between Profile and MyVA. If you had time in wrap up questions you could consider: Ask participant how they would describe these two pages to another Veteran.
• The Mural co-design could be an interesting experiment. I would update the desktop design to mirror the CodePen prototype, specifically the Sign out button being removed and sign out link added to bottom of auth menu. We know that with OFCIO guardrails having this separate sign out button on desktop is a no-go.
• If you have any questions, let me know. Happy to help clarify anything as you go through your research prep. Looking forward to observing a few sessions!

[cindymerrill]

FYI In my experience, Shane doesn't approve research for recruiting without a working prototype. @andaleliz @aklausmeier

[andaleliz]

@aklausmeier thank you for this feedback, and linking to that CAIA a11y review ticket. This is all really helpful. I also appreciate your willingness to be available to answer any future questions we have too!

Responses to your questions/suggestions:

curious if you are planning to kick off prior to your team's revised prototype being ready for Shane's review?

Yes, we are planning on kicking off recruiting before the prototype is totally complete. We expect to have it far enough along for Shane to understand what we're doing and be able to start recruiting. I'm not sure Shane will go for it but I think it's at least worth a conversation with him (cc @cindymerrill).

Feel free to pull OCTO and OFCIO research priorities from our research plan and anything else you might still need. Might be good to link back to our research plan as your research is fulfilling "Phase 3" that we reference in our research plan.

Great idea, and thanks for pointing out these needed additions. I didn't realize that we were conducting the Phase 3 research you had planned; I was under the impression that this was a preliminary supplement to what your team would do in phase 3. I've updated our plan accordingly.

In our research we discovered that folks voiced confusion between Profile and MyVA. If you had time in wrap up questions you could consider: Ask participant how they would describe these two pages to another Veteran.

I love this question, and have added it to the post-task interview portion of the convo guide.

The Mural co-design could be an interesting experiment. I would update the desktop design to mirror the CodePen prototype, specifically the Sign out button being removed and sign out link added to bottom of auth menu. We know that with OFCIO guardrails having this separate sign out button on desktop is a no-go.

Yes, I'm interested to see how it goes. I removed the sign out button from the desktop design, and added Sign Out to the bottom of the list of links. I also removed the x from the Close button on mobile, since I don't see that in your prototype, and added Sign Out to the bottom of that list of links too.

Closing out this ticket since we have no further revisions to our plan.

[Samara-Strauss]

I didn't realize that we were conducting the Phase 3 research you had planned

I'm sorry -- that is on me for not having communicated this adequately.

[andaleliz]

No worries @Samara-Strauss! There was a bit of telephone going on since we were both out at different times, and the holidays. It feels like we still got a place that will cover the needs for Phase 3, even without that specified from the get go 😄