[Discovery] Logged-in homepage: User research — Synthesis

[Samara-Strauss]

Background

Epic #7268

As part of the logged-in homepage discovery phase, we have decided to conduct user interviews/testing to help us better gauge user needs and expectations.

Tasks

• [x] Write research report
• [x] Edits
• [ ] Share out to team

Acceptance criteria

• [ ] Report has been written an shared with the team

[andaleliz]

@sshein, @Samara-Strauss, @tressaellen I've completed the topline summary and would love to get your feedback on it before I dive into the detailed synthesis. Open to all comments but am specifically looking for feedback on:

• Do these findings line up with what you observed in the sessions?
• Anything missing?
• What topics would you like me to focus on in particular (if any)?

Could you please let me know your thoughts by 9am EST tomorrow? If you need more time, or it would easier to discuss in a meeting, let me know. Thanks!

[Samara-Strauss]

cc/ @mshea0606 ☝️

[Samara-Strauss]

Hey!! This is awesome!

• In the top section with the other research docs, don’t forget to include the blue sky prototype in the full summary. In case it wasn’t already on your radar, I want to make sure it gets into the final report.
• "Participants wanted to see current information related to their benefits on their personalized page. Healthcare and disability benefit items were prioritized by participants. Secure messaging, appointments, prescriptions and accessing documents or letters were brought up in multiple sessions.” —> Don’t forget claim status and specifically what disability information people wanted to see. It might be helpful to list the popular content people wanted to see by benefit so this info is easier to scan and has more emphasis. This is definitely one of the most important questions we answered with research so I don't want it to get too buried.
• "Multiple participants expressed wanting to see additional information that would help them utilize benefits available to them and take better care of themselves on their personalized page: adjacent benefit information (e.g. disability compensation recipient wants to learn about Voc Rehab), self-care/wellness, community information, etc.” This is awesome and I think something we should also bring more emphasis to. In the actual report, can we be more specific about "self-care/wellness, community information, etc.”? Idk if some list would also be helpful here, but it might be.
• "Participants talked about the current experience as being overwhelming or following a lot of links to get to where they're going, but when designing their own page, did not mind direct links from their logged-in homepage to the task.” For the full summary, it would be helpful to explain what “the current experience” is referring to.
• "A design like the blue sky prototype would be useful to Veterans.” I would be more specific about what people found useful. You do mention the “snapshots” — I think we can go further and say that, specifically, participants wanted those sections on their dashboards if those benefits applied to them, and also that many of these sections were self-generated by participants for their own dashboards. I would also mention the navigation. At least in the sessions I sat in on, people loved that the side nav wasn’t buried, and the IA of that section matched their expectations.

Things that don’t fit quite as nicely into the original questions/hypotheses:

• I think one of the biggest findings from this research is that it validates people want to see information that is relevant to them when they log in. This completely validates the work we’re doing.
• The above point also underscores that, while popular, health care and disability information shouldn’t make us blind to other benefits. At least in the sessions I sat in on, a couple people mentioned edu/GI bill benefits and information on home loans (or having used VA home loans!). I think the key here is, again, that we should feature the benefit information that is relevant to a particular user, and we have to be able to scale the logged-in homepage experience in a way that accounts for many different scenarios. Health care and disability may be king, but we recognize these are not the only parts of the veteran experience.

[sshein]

cool, looks good. :) Moving into your final synthesis, I would think about "What are the top insights that would be most useful for a designer coming into this project"? What are the quotes you'd really want them to see?

[Samara-Strauss]

Heavy +1 to everything Shawna said.

[andaleliz]

Thanks for your feedback @Samara-Strauss and @sshein. All makes sense and great point about the new designer. Synthesis is in progress!

[andaleliz]

@Samara-Strauss question for you:

Don’t forget claim status and specifically what disability information people wanted to see. It might be helpful to list the popular content people wanted to see by benefit so this info is easier to scan and has more emphasis.

I think you're right on with the list idea. I understand Voc Rehab to fall under disability benefits. Do I have that right, or should it be it's own benefit category in the list. It's currently structured like so:

• Health: [list of items]
• Disability: [list of items, including Voc Rehab]
• Education: [list of items]

[Samara-Strauss]

I think you're right on with the list idea. I understand Voc Rehab to fall under disability benefits. Do I have that right, or should it be it's own benefit category in the list. It's currently structured like so:

OK, so I think we can leave it that way because you have to have a disability rating to qualify for voc rehab and people associated it in the research with disability benefits. BUT, technically I think we consider it a "careers" benefit, at least with how it is categorized on the website.

https://www.va.gov/careers-employment/vocational-rehabilitation/eligibility/

Still, since veterans in the research specifically talked about it in the context of disability benefits, I think the categorization you have works just fine :)

[andaleliz]

@Samara-Strauss the findings summary is ready for you!

Note: I didn't include a recommendations section in this document; rather provided implications/takeaways for each finding. I started to write them up, but it felt like it would be redundant based on the recommendation-focused overall discovery summary we'll collaborate on. I'm happy to add in a specific recommendation section if you feel that it needs it once you review the report.

Thanks in advance for taking the time to read it, and I look forward to your feedback! :)

[Samara-Strauss]

Hey @andaleliz!! Great work! This is super comprehensive. My feedback most has to do with the key findings.

Also, re: recommendations vs. implications — I think what matters is more the content than what we called them, and your takeaways for each item felt right on.

Key findings 1 & 2: "Our logged-in homepage will meet Veterans needs.." and "Participants want a page that is focused on current information..."

IMO, these are too similar. I would eliminate 1 entirely — I think you say everything that needs to be said nice and succinctly in key finding 2 (“Participants want a page that is focused on current information…”). If you want, you could add the quote from 1 so we don’t lose it.

Key finding 3: Participants primarily log in to complete healthcare and disability related tasks such as messaging, ordering medication, and checking claim status.

• I would reword this slightly to put more emphasis on that participants wanting their dashboards to be task-oriented. Then I would mention that for most users, this meant tasks related to managing health care or disability-related information.
• This is small, but for the disability stuff, I would put claim status at the top of the list since that seemed like the most important thing to everyone. Also, I don’t have a sense of this, but did you feel like there were primary/secondary categories for what was important for the disability stuff? I definitely would have emphasized claim status, rated disabilities, file upload, and info on claim for increase over info on voc rehab and housing assistance. But idk if that is my own bias, so feel free to leave it as a one-tier list if that feels most true to the results.

Key finding 4: “Participants wanted to see information on education benefits on their personalized homepage.

I would reword this because it sounds like everyone wanted edu benefit info. You go on to clarify, but the heading itself is misleading. Maybe “For those interested in education benefits, GI bill balance and payment information was important.”

Key finding 6: Participants were interested in information that helps them maximize their VA benefits, be more involved with the VA as a whole, and take better care of themselves.

• Can we reword this to specifically call out that people want info on benefit recommendations or eligibility? I think that’s clearer than “maximize their benefits.”
• Also, I would consider breaking this into two findings — one about benefits eligibility and recs and one about veterans wanting info on events and resources at the VA. This one is kind of long and I think we could address these two points separately, but if you feel the content goes together, let’s leave it as-is

Key finding 8: Benefit information and personal information should be separated on the logged-in homepage, with benefit information taking a higher priority.

For me, the bigger conclusion here is that people didn’t verbally express a strong difference between benefit and personal info, but they did regularly group “profile” things together. Also, the other big thing is that almost half of the participants did want to see or have access to this info from the dashboard, so we may want to consider this when revisiting the authenticated IA/navigation.

So, maybe that leaves us with something like "Participants don't see "profile" information as totally separate from their benefit information. Some participants even wanted to see or have access to "profile" information right on their logged-in homepage."

Key finding 10: A design like the blue sky prototype would be useful to Veterans.

Would be super helpful to have a screenshot of the main page of this prototype here. Doesn’t need to be the full width of the page like you have later on — could just be the size of what you can see on a laptop screen.

[andaleliz]

@Samara-Strauss the findings summary has been updated - thanks for your feedback. It feels more complete now and I think folks will get a lot out of it, even if they were to only skim the key findings section. A couple of comments:

This is small, but for the disability stuff, I would put claim status at the top of the list since that seemed like the most important thing to everyone.

I put rated disabilities up at the top, because that was the first thing across the boards for folks who had this benefit. There were a couple of disability recipients who had claim status as a secondary priority, mostly because their disability is "stable" and they don't typically have pending claims (at least as I understood it). One participant with a 100% disability rating put claim status as her 2nd priority in her disability section, but then verbalized that she understood this to be all claims she or her spouse might file with the VA, and it wasn't just disability. I added a little clarification on this to the report.

So, maybe that leaves us with something like "Participants don't see "profile" information as totally separate from their benefit information. Some participants even wanted to see or have access to "profile" information right on their logged-in homepage."

We still had 4 participants who did separate it out from benefit info AND weren't interested in it at all, so I just added "Most" to this finding to ensure if someone skips the details, they're not left with the impression that this applied to everyone.

I like the addition of the blue sky screenshot to finding 10 - added one to the notification finding for clarity too. Great suggestion!

Please let me know if you'd like to see any other edits to this report before we close out.

[andaleliz]

Also - would a comprehensive deck of these findings be helpful? VSA research process suggests to create a comprehensive deck for sharing out with the larger team.

I'm thinking we'll do that for the final summary, so not sure we need both. Happy to create one if it's useful!

[Samara-Strauss]

I put rated disabilities up at the top, because that was the first thing across the boards for folks who had this benefit. There were a couple of disability recipients who had claim status as a secondary priority, mostly because their disability is "stable" and they don't typically have pending claims (at least as I understood it).

Noted! Thanks for clarifying. Totally understood :)

We still had 4 participants who did separate it out from benefit info AND weren't interested in it at all, so I just added "Most" to this finding to ensure if someone skips the details, they're not left with the impression that this applied to everyone.

Perfect. Sounds good to me.

Health care beneficiaries prioritized the following items: VA Medical Center

Is this their primary medical center, or finding other medical centers?

There were less common benefits that participants brought up in sessions such as VA home loans and life insurance. Participants that had these didn't prioritize them on their homepage, but the fact that they discussed them in relation to a recent visit to VA.gov suggests they are still important to them.

This is under the GI bill benefits section, but there's nothing about other benefits mentioned in the key finding. I would either add another key finding to address benefits outside of health/disability/edu or add something in the key finding to address other benefits. Otherwise, I think this information could get lost.

---

Otherwise, this is really, truly SO awesome and I am so excited to share this with the larger team. GREAT work!!

[andaleliz]

Is this their primary medical center, or finding other medical centers?

They were referring to their primary medical center - added clarification

This is under the GI bill benefits section, but there's nothing about other benefits mentioned in the key finding. I would either add another key finding to address benefits outside of health/disability/edu or add something in the key finding to address other benefits. Otherwise, I think this information could get lost.

GREAT catch - this was hanging around in there from one of the early versions of that finding (I had reworked it a few times before sharing out). I moved that chunk to the Additional findings section since participants didn't really put that on their personalized page, but did discuss it in the intro interview and later in the sessions.

I also added some more context to the education benefit finding details, including a recommendation to include younger veterans in future studies for this effort. We know that Edu benefits are lower priority than health/disability, but it'd be good to validate that with younger folks since all of the under 34 Veterans we had scheduled were no-shows.

[Samara-Strauss]

Thanks for the update. Just gave the updated sections a glance, and they look great. No other changes on my end. SHIP IT!!! 🎉