Design | Profile Hub | Convo Guide

[mtcA6]

Background

We need to create a convo guide to support the Profile Hub

Tasks

• [x] Draft conversation guide
• [x] Review with PM/PO
• [x] Incorporate edits

Acceptance Criteria

• [x] Guide complete and ready for submission

[pamela-drouin]

@mtcA6 carrying over 3

[pamela-drouin]

@andaleliz ready for your initial review! It's a bit free-wheeling in that depending on how each conversation kicks off, I may ask the following prompts/tasks in a different order. It makes sense to me, but let me know if it makes sense to you.

[andaleliz]

@pamela-drouin great job! I see what you meant about "freewheeling" but I can absolutely see why this will work well for this particular study. This is really exciting to see and I really appreciate you asking for my feedback!

I have a few comments/questions/things to consider:

Warm up

• In the warm up, you may want to skip the general "tell me about yourself" to keep the questions focused on things that will be useful to the research. I've found sometimes people can be very chatty, or it ends up feeling awkward because they shared all this stuff and then ultimately my response is "Ok, thank you, moving on" 😆 but totally fine to leave it if it makes you more comfortable or you've had good success w/ this in the past.

• for "walk me through what you had to do", it's not clear to me if you want them to tell you, or sign in and actually walk you through it on the website

Various tasks

• These feel focused on navigating to tasks from the hub page, which will be very helpful in answering our research questions regarding the lack of the Profile sub-navigation. However, I wonder how effective they'll be at answering our other research questions about the hub page:
  • How effective are the section overviews in giving users the gist of what’s within each one?
  • Do participants resort to using Search to complete any tasks?
  • How useful are the deep links? Are they confusing?

I know you have follow-up questions listed separately so maybe those will help get us these answers! But, if I were facilitating I would add some specific things around this to be sure I didn't forget. That's just me though.

• Are you going to have people go through all of these, or just pick a certain amount depending on what they say?
• I don't see guidance for helping people log in; if they aren't doing it during the warm up, you may want to add that into the beginning of this task as a starting point. Unless you're purposely not starting them on the website.
• I love that you are offering more opportunities to share their experience "If you’d like to update any of this information while we’re on this call, we’d be happy to learn from your experience." ❤️
• Awesome job phrasing the tasks without using leading words! I know that was challenging.
• The follow-up questions felt like they were missing, but then I see you had them in a separate section. I typically put them immediately after the task/question so that I don't miss anything or forget since sometimes I get overwhelmed and/or tired in a session

Wrap up questions

Is there anything in your Profile you expected to be there, but isn’t?

I had to read that a couple times - I think what you're trying to get at is: "Is there anything you expected to be in your Profile, but isn't?" or "Is there anything missing from your profile that you expected to be there?"

One final comment about the overall doc is that we frequently put a "things to watch for" section for each task/question to highlight what we're trying to elicit. This is helpful for observers/notetakers, but I have also found it helpful as a facilitator to help me cross my t's before I feel it's OK to move on to the next task or question. If you don't include something like that here, you may want to create some other kind of artifact to help your notetaker (I know that's TBD) or observers focus on things to highlight.

Again, great job!! Woot woot! 🎉

[pamela-drouin]

@andaleliz this is the best feedback I could ask for. Thank you, I can't wait to make another round of edits!

[andaleliz]

@pamela-drouin I spoke with Jonathan Nelson today regarding Nok and EC, and he thought it would be really helpful if we could try to elicit some feedback on where people would expect to find that in the profile. What do you think about adding something around that at the end of the session? I personally would be interested to know:

• where do people expect to find that in the profile?
• what is their understanding about how VA uses Nok and EC information, especially if they don't have health benefits?

[pamela-drouin]

@andaleliz love that! this definitely feels perfect for the end! I will include some questions around these topics in my edits (should be sharing it out today) 😃

[pamela-drouin]

@andaleliz I updated the convo guide significantly. I put a few questions to you in {squiggly brackets}.

The formatting is a little intense, and it's kind of hard to keep track of the two possible flows when the tasks start. I would love if you put your "colleague won the lotto so I gotta facilitate this session" hat on and give me all the feedback! 🤠

[andaleliz]

@pamela-drouin WHOA! This is awesome! I love how you used some wingdings as a cue for the path to take! That is super smart and I picked up on it right away. Overall, the guide is incredibly clear to me and I think I could jump in and facilitate the session tomorrow if I needed to.

---

Answers to your questions

If needed: [add zoom help?] [additional help text around logging in – example?]

As the colleague of a lottery winner, I wouldn't need this added into the guide since I've done it so many times. If you think you can walk some through it without having it written out, I'd leave it out.

{pause here to ask for a participant’s first impression of hub? good to get initial impression, but feels artificial and may cause someone to look more closely than they otherwise would, effecting time on task for subsequent tasks}

I'm so torn about this! I think you're right about it feeling artificial and potentially skewing behavior. Would you feel OK with leaving it out for now and seeing how the pilot session goes? Sometimes as you go through it out loud with another person, the right thing to do becomes very clear. I think we could potentially elicit their overall impressions as part of task 5, or the post-task interview if it doesn't come up organically.

What tipped you off that you’d find that information here?

I love this phrasing! Definitely going to borrow that.

{Include a disclaimer that right now NOK/EC is only for healthcare?}

As far as I know, we haven't confirmed that is actually true. I'm still waiting to learn more about this from VBA folks.

---

Other feedback/notes

• I really like how you formatted out the intro section, that makes it a lot easier to follow. Especially when accounting for AT/magnification/non-AT sessions all at once.
• In the task 4 prompt, I think there's a you missing: "Let’s pretend you just switched banks, and you need to update where get your payments from VA.". But, I don't know why but thinking about a "where" in that scenario feels weird to me. I wonder if something like "Let’s pretend you just switched banks, and you need to update information so you can still get your VA payments" would be a more direct (without being leading)
• Smart of you to add how they navigated to the "things to note" sections! Thanks for calling that out.
• Just FYI: When I estimate timing for tasks, I usually tack on an extra 1-3 minutes from what I think, just to have buffer time.
• If the session feels long after you run the pilot, we may want to cut out one of the tasks - I think 7 would be the best candidate.
• I think Angela Fowler would be glad to run a pilot session with you if you want to practice with a screen reader user, in addition to the normal pilot.

Great job! I'm so excited for this research. 🎉

[mtcA6]

@andaleliz I asked Pam include the disclaimer language about NOK/EC since the work completed thus far by the MHV team focused on NOK/EC for healthcare ONLY. Mike and team said as much on our call last week.

If you get answers from VBA folks about this spanning multiple benefits then we can drop disclaimer language, but currently the MHV folks have focused only on the health care "business line" and we should reflect that.

edit: confirmed with Mike Collier the API they're plugging into is health care specific "We have confirmed that the Health Benefits API is healthcare only. It updates the VHS enrollment system, and ultimately ViSTA instances."

[andaleliz]

@mtcA6, sharing my thoughts below but ultimately, I think this is Pam's call as the researcher on what she thinks is most appropriate. Whatever she wants to do is good with me :)

• That all makes sense since the MHV is a health care focused team and that's their priority. Since our team is benefit agnostic, I think we need to do our due diligence and thoroughly understand the landscape of NoK and EC across VA before we tell research participants anything definitive about how these things work, or even before we make decisions about the functionality in our product.
• Since this is a generative research question, I'm not sure what the benefit would be to share a disclaimer that Next of Kin and Emergency Contact info are only used by VA for health care purposes. Are you thinking that's going to influence where the person would expect to see it in the profile? I could see it being important to clarify this with participants if we were usability testing the actual NoK/EC parts of the site, but at this point, we're just gleaning insights about their mental model.
• If a person has ever been to a VAMC, they'll have a record in the health benefits API. Some people only go there to be examined in relation to a claim for disability compensation, and never go back. So in their mind, they don't use VA health care, but they'd have a record in the API and potentially NoK/EC information.

[pamela-drouin]

@andaleliz I always love getting feedback from you -- thank you so much! I'll be making some edits for sure 😁

@mtcA6 I am less interested in communicating up front how things are currently technically handled. I am more interested in finding out first what people’s perceptions are regarding how EC/NOK information might be interpreted and handled by VA. These kinds of insights may indicate a gap in understanding vs. actuality that would be valuable for us and other VA.gov teams to learn about and address.

If we need to manage expectations around what VA.gov does actually do with EC/NOK, I would be happy to state that after we’ve asked questions that generate people’s perceptions. I would need some help on wording something around:

Thank you for your input! Currently VA.gov handles EC/NOK information [in this particular way for this particular benefit], and we are gathering input from multiple veterans to figure out ways to better support everyone's needs.

[Samara-Strauss]

Chiming in on the NOK/EC issue. I don't think it hurts to do the following:

• Ask "Where would you expect to be able to list your Emergency Contact and Next of Kin in your Profile?" as it's already outlined, AND
• Ask if this information was only used for health care and not other benefits, would participants expect NoK/EC to be handled differently or in a different location.

I think we can be both comprehensive and also gather information about the very real waters we are currently swimming in at the same time. But final call is up to Pam.

[pamela-drouin]

I made edits last week -- anyone up for a final review before we deem this ready for submission?

[andaleliz]

@pamela-drouin I have time today and will be glad to give it another review! 😁

[andaleliz]

This is looking so good! Just a couple of small things:

• We'll need to add estimated times per task before submitting
• In task 6: the phrase "This link took you to another page – how did that jump feel?" should maybe say "This link took you to another part of the website – how did that jump feel?" since all the links actually take people to other pages.
• In task 7: what do you think about adding "how did the participant navigate back to the hub" in the Things to Note section? That'll be interesting since they'll be going back from another part of the website (rather than from within profile like the other tasks)
• In Emergency Contact and NoK questions, I still don't think you need to give a verbal disclaimer but up to you! I would either remove your note from the guide, or include what you'll say prior to submitting for review.

Kudos to you Pam, this is so solid! Can't wait to observe the sessions.

[pamela-drouin]

Thank you so much Liz! I sure did forget to estimate the time per task last week! I'll adjust everything else per your suggestions.

[pamela-drouin]

Yesterday, I updated the guide with time estimates. This morning, I added some verbiage to cover whether a participant clicks within a card, but not on the link itself.