JUNE: User Testing

[NealHumphrey]

On June 13th, we will launch the V1 of our website. The User Testing group is responsible for setting up a system for user testing and have it ready to conduct user tests just following the launch (June 14/15) so that we can quickly turn around feedback to incorporate into the V2 launch 6 weeks later.

• [x] Determine goals, usability questions, and user stories (by May 16th)
• [x] Create user testing script (by May 30th)
• [x] Determine location and methodology of user testing (by May 30th)
• [x] Set up feedback mechanism (By May 6th)
• [x] Schedule interviews (By June 13th)
• [x] Conduct user testing sessions (Week of June 20th)
• [x] Write-up evaluation summary (by June 24th)

[txomashernandez]

Running list of user stories to test, to be revised as the filter and browse views are defined:

• As a Housing Insights user, I can search for a specific subset of house based off of certain criteria and pull up a list of just those houses.
• As a housing Insights user, I can browse for a specific set of houses that meet given criteria.
• As a housing Insights user, I can select a zone type (ward, zip, etc.) and a data set from a large list of data sources, and view a choropleth map of results.
• As a housing Insights user, I can clearly tell which filter criteria have been applied.
• As a housing Insights user, I can tell the meaning of color coding of dots on the map (i.e. buildings).

[txomashernandez]

Running list of testing goals and usability questions, to be revised with the further refinement of the filter and browse view's development:

User Testing Goals:

• Validate the design direction of having a combined browse and filter sections.
• Determine if interesting and useful housing data is displayed and searchable.

Additional Usability Questions:

• Are specific elements confusing or easy to find?
• Do you get the information you need?
• What would you expect to find that you didn’t?
• Are you able to filter down in a useful way?

[txomashernandez]

Testing Script: https://docs.google.com/document/d/18Iv6NWktWU-DNj8pYjcbKGoTQbLoTlmpn2YHla9926E/edit?usp=sharing

[txomashernandez]

@NealHumphrey - Getting close to the testing session, very excited! A few quick questions for your before tomorrow's planning session: 1. Location: Do you have any thoughts on the location? If not the usual spot or somewhere you've got in mind, we could use my office. It's actually right across the street from the weekly session meeting building. 2. Participants: I think you have a list of people that should be a part of the user testing. Do they need to be contacted or invited? 3. Feedback Mechanism: In the past I have used screen recording software to record the session, but don't have access anymore to anything too powerful. I have used my Webex meeting screen recording with no problems, which would allow us to share with others in another room if we wanted and keep a recording of user site behavior. 4. Testing Scenarios: In addition to logistics, tomorrow we can finalize a list of scenarios to walk through with the participants. I had started a list (in the script doc) based off of old conversations but look to you and the dev teams for a good list of scenarios that would be useful to navigate through.

A few more questions come to mind, but we can discuss tomorrow!

Thanks in advance, Thomas

[NealHumphrey]

Hey @txomashernandez . Here's my thoughts:

1. We can use our regular classroom if we schedule any of them during the Tuesday work session. I think it would be nice to aim for holding 2 user tests on June 20th and set it up with two class rooms so that people can observe and then discuss results afterwards, but we probably only have time for 2 there. I'd say if you and any other testers can schedule 3 more tests (for 5 total) based on your + the testers schedules then location can be up to you - your office would be great if it's possible. You can especially be sure to record those 3 so the team can watch the tests in addition to any summary findings. Only limitation is I have a family trip that week so it'll need to be all up to you and the team.

2. I think for this round we use people we interviewed. Yes, I have a list and we do need to contact them still. Let's coordinate how best to do this based on the decision on location+schedule.

3. Awesome, yes definitely record, and for the 2 on Tuesday we can set up one classroom with the tester and the other with observers.

4. The biggest thing I want to test is the no-scenario version - what will they do if we just say 'start exploring'. Will they find any interesting stories? Will they be confused as to the purpose? Will they get excited about using it more? This is the most realistic scenario - i.e. when we launch the final tool version, there will be an email announcement and some event, so they'll know about the tool - if we aren't there to suggest things to them, will they find value in it?

We can also work through a list of specific tasks to have on hand if they don't naturally do any of the steps we hope to test.

[txomashernandez]

Neal,

1. That plan sounds great. We can plan for 2 on the 20th, and the other 3 when it makes sense for the other participants. My office at 1152 15th Street (Across for the Code Academy) should work.
2. Sounds great, will discuss how we want to schedule those this evening.
3. Excellent, we'll plan on recording all sessions.
4. Yes that sounds great, we can lead with that and then go into a few specific scenarios after the open exploration.

Thanks! Thomas

[txomashernandez]

@NealHumphrey here's a doodle with my availability for the week of the 20th: https://doodle.com/poll/3gxrfzfgy9m3zmkh. Feel free to use in the intro email to potential participants.