Create Documentation for User Tests

xiuda commented 6 years ago

Problem / motivation

On the first day of Week 5, users (ie your classmates, TFs, or Ken) will be testing out your project for bugs and usability. Create a document to outline how they should test your program.

https://www.userzoom.com/user-experience-research/a-simple-template-for-writing-usability-tasks/ https://www.nngroup.com/articles/task-scenarios-usability-testing/

Note: You might want to look up guides for game-specific user testing

Document Requirements

See the details section for more info.

[x] Describe the User/Role your testers will be playing
[x] Write out a list of User Goals you want to test
[ ] For each user goal, write out a "scenario" the user should try to accomplish
[ ] Write down how to do each task (but don't share this with the user)
[ ] During the session, take notes of how well the user was able to do your task without any guidance from you
[ ] Write 10-12 tasks overall

Details

User/Role Description

Provide enough context for the user to be able to do their task. For example, for a flight booking website: You're planning a vacation to New York City, March 3 − March 14. You need to buy both airfare and hotel. Go to the American Airlines site and jetBlue Airlines site and see who has the best deals. Who is the user and why do they want to be interacting with your application?
The user should be able to know everything they need to complete each task from this description you give them.

User Goals

Make your goals realistic
Don't make them too specific to a feature. Think about what a user would want to do with your app.
Think about your user stories from earlier

As a customer, I want to be able to look up flights to my desired location from my local airport As a customer, I want to compare prices of my flights As a customer, I want to purchase my tickets.

Scenarios

A good scenario leaves a little room for the user to discover what they should do. You should not have to tell the user how to do it, it should be obvious from your program how to do each of those things. If your tester cannot figure it out, give some helpful hints, and record where they got stuck and why, so you can make better design later.

Make the task realistic
Make the task actionable
Avoid giving clues and describing the steps

Help the user find their way, but don't give them the answer.

Scenario Comparison:

[Bad] Purchase a pair of orange Nike running shoes (not realistic for all users, doesn't give user much freedom.)
[Good] Buy a pair of shoes for less than $40
[Bad] You want to see a movie Sunday afternoon. Go to www.fandango.com and tell me where you’d click next.
[Good] Use www.fandago.com to find a movie you’d be interested in seeing on Sunday afternoon.
[Bad] You want to see the results of your midterm exams. Go to the website, sign in, and tell me where you would click to get your transcript. (Lists out too many steps)
[Good] Look up the results of your midterm exams.
[Bad] Make an appointment with your dentist. (Too vague)
[Good] Make an appointment for next Tuesday at 10am with your dentist, Dr. Petersen.

During the usability test

Usability tests do not have to be a user sitting quietly in a box and doing tasks for you. They are welcome to think aloud and you are welcome to ask each other questions. Here are a few guidelines for conducting a usability test:

Write down as much as possible! Have 1-2 people be the "conductors" and everyone else be scribes.
Don't jump in and tell them what to do unless they get 100% stuck or encounter a bug. Write down where they got stuck and what the bug was
Answer questions if they ask
If they do something you didn't expect, ask them why they did that
Write down the alternative path a user took, if it was different than what you wrote down.
At the end, have everyone compare notes.

xiuda commented 6 years ago

Instead of a more formal application-based usability approach, you might want to have people play through however many levels you made and think of what questions you want to ask them at the end. https://www.surveymonkey.com/r/?sm=ubUCfsRg7sSmVfqRVQ3AxA%3D%3D

karlin88 commented 6 years ago

https://github.com/alyssalina/catsandbeer/wiki/User-Testing-Documentation

alyssalina / catsandbeer