search

department-of-veterans-affairs / va.gov-team

Public resources for building on and in support of VA.gov. Visit complete Knowledge Hub:
https://depo-platform-documentation.scrollhelp.site/index.html
283 stars 204 forks source link

Candidate Address - Override/ User Testing Conversation Guide #2327

Closed artistgreen closed 5 years ago

artistgreen commented 5 years ago

Goal

Definition of Done

artistgreen commented 5 years ago

Conversation Guide

Justin-Pickett commented 5 years ago

@artistgreen has completed the conversation guide. Next steps, he will share this with @Samara-Strauss and @lkoenigsberg for their review and feedback.

When the review has been completed and feedback has been offered please check the the button below.

Samara-Strauss commented 5 years ago

@artistgreen sorry I'm just getting to this now — I must have missed the original message from Justin. I do have a lot of feedback — this needs a good bit of work, so let's get this in good shape before EOD tomorrow so there is time for review before Friday morning's session. If it's not ready, we'll have to postpone the sessions.

Overall feedback

  1. I would fully build out the prototype so we can click through it before Friday morning. It wasn’t fully clickable so I had trouble reviewing it.

  2. Overall, the script needs a bit more narrative. Perhaps it’s because the prototype wasn’t done, but I didn’t have a sense of how you wanted the participant to move through the flow. I personally find it helpful to include transitions between sections and tasks so I’m not fumbling over my words in session, but this isn’t for everyone so do what feels best for you.

  3. Also, the script doesn’t give you much room to evaluate the user’s understanding. There are a lot of commands and not a lot of asking users what they think they should do or how they understand what they are looking at. This needs work so we can make sure we’re fairly evaluating the design instead of just telling the user what to do (which will of course make the design perform well).

  4. Finally, watch for leading questions. There are a couple instances where you ask if things are clear/quick/easy. Asking a participant “Do you find this to be [x]” will prime them or make them feel obligated to say the design is “[x]”. To avoid leading questions, ask things like “What do you think this means?”, “Tell me about that experience,” “How does this make you feel?,” etc.

Prototype

  1. Use a whole page mockup for the profile. I can send this file if you do not already have it. It feels too much like cheating to cut the page off like this. Limiting the user’s scope this drastically will of course lead them to the right answers, and it won’t accurately give them a feel for what they would experience on the site in real life.

The script

  1. Protip— If the user is having trouble sharing their screen, you can have the prototype on your computer and share keyboard and mouse controls with the user so they can manipulate the prototype from their end.

  2. “Have you updated your mailing address at any time?” — May want to clarify this is at the VA.

  3. “Were you able to update your mailing address easily and quickly?” This is a leading question and needs to be removed. I would ask “Tell me about that experience” as a follow-up to “Have you updated your mailing address at any time at the VA?” if they say they have updated their address.

  4. "Begin on dashboard screen” — I would maybe consider starting them on the homepage, but I do like that we are not starting them right on the profile!! It's great to evaluate if they even know where to go to update their address. Also, I did not see the My VA dashboard (or homepage) in the prototype — it would be helpful to get that in the prototype so we can click around it before the first session Friday morning.

  5. "Please click through the form, as this is a prototype, we won't have you input any information.” Two things: (1) I would explain in the introduction that this is a prototype to see if they have any questions about what a prototype is or why they are not seeing their own information. (2) This language is all insider baseball. A participant will not understand “click through the form” or what being a prototype has to do with them not putting in information. Use plain, specific language that explains why they can’t click into the form field and then ask them to specifically click “Update.”

  6. “This screen has an alert - is it clear why we show the alert Is the alert messaging clear as to what is happening?” Asking if something is clear (or good or bad or whatever) is a leading question because you are priming the participant with by thinking of the design a certain way. This needs to be reworded to something like “Why do you think we should this alert? What do you think it is trying to communicate?”

  7. “Please select an address option to continue” Instead of telling the user what to do, I would ask them what they think they should do next so you can evaluate the design. It is OK to tell people what to do in UAT or something where we are not evaluating usability, but since we are trying to gauge understanding here, commands are not a good idea.

  8. "Update your mailing address” Again, I wouldn’t give them commands. I would ask them “How would you do X?” So you can evaluate whether they understand the design or not.

lkoenigsberg commented 5 years ago

@artistgreen I'll defer to @Samara-Strauss on most of this except for the fact that the prototype does not seem to be a true reflection of what we are testing. I only saw the "happy path" where a user updated and all went well. Where are the alternative suggestion paths? Probably just something I'm doing wrong but it's got to work and be clear.

Samara-Strauss commented 5 years ago

@lkoenigsberg I will say it's hard to test all the alternative paths with an InVision prototype since we'll only ever link up a certain number of screens. That being said, things that I find to be helpful:

1) Always have full pages. No half page. 2) Maybe start them on the homepage or some neutral page and have them figure out how they would get to the desired destination. We don't need to go too over the top here since that's not the main objective of testing, but anything that simulates production will help to put them in a similar mindset to what they would be experience IRL. 3) Make sure Cancel and X buttons are always linked, and don't ever just link the one thing the user needs to click. Then they can cheat by clicking elsewhere on the screen and seeing what InVision app highlights to find the right path. There should be multiple things people can do on most screens.

Samara-Strauss commented 5 years ago

Update: I will be working on this Monday/Tuesday of next week and post an updated convo guide and prototype here. Testing is targeting for 11/13 – 11/15.

Samara-Strauss commented 5 years ago

@artistgreen @lkoenigsberg @Justin-Pickett Here are the updated prototype and conversation guide. Please let me know if you have any feedback. I'll incorporate it when I get back. Testing is scheduled to begin Wed, 11/13.