Test updated election summary designs with data users

[JonellaCulmer]

@patphongs commented on Mon Mar 12 2018

Referencing issue #106 from fec-testing as that pertained to testing election summary pages from the recent usability testing in Scottsdale. After synthesis, there are a few design tweaks that we should test further.

• [x] @JonellaCulmer Please provide @patphongs with the updated sketch designs for the election summary pages so that we can code this up before 5.3 testing.
• [x] schedule timeslots on 18F appointment calendar Thursday and Friday
• [x] contact data users for testing
• [x] write test script
• [ ] open follow-up ticket(s) for further updates and/or implementation

[JonellaCulmer]

A prototype of these update election summary pages can be found here: https://fec-feature-cms.app.cloud.gov/data/elections/house/AZ/01/2018/

The updated designs include modified cards, added icons for improved visibility for the reporting buttons, and modified language that explains what data is included in the candidate financial totals table.

Updated designs are below:

About this election page

Election data and reporting deadlines page

[JonellaCulmer]

Testing script for update election summary pages.

March 22-23, 2018 Test Script

Quick connection links:

Notetaking: https://docs.google.com/spreadsheets/d/1Z3T45hb1WAhD1zRL_fLXZlfQ9ApenTvgYE0OM0eLhaE/edit#gid=313597352 Mural board: TBD

Features to test:

Election summary race pages

Tested on prototype: https://fec-feature-cms.app.cloud.gov/data/elections/house/AZ/01/2016/ Visibility of reporting/compliance buttons Usability of race page

Introduction script:

Hi, I'm ___, and I'm working on the Federal Election Commission website, and we'd like to get some quick feedback on new features we're adding. Before we start, I want to make it clear that we're testing the designs, not you. You can't do anything wrong here and all your answers will be the right ones. Also, please don't worry that you're going to hurt our feelings. We're doing this to improve the site, so we need to hear your honest reactions. Although it may seem a little weird, I'm going to ask you to think out loud as much as possible as you look at the designs. Describe what you're looking at, what you're trying to do, and what you're thinking. This will be a huge help to us. Sound good? [wait for user confirmation; clarify things if user asks questions] (credit: Krug)

Introduction questions:

1. What is your job in campaign finance?
2. Do you visit FEC.gov often?
3. When you visit FEC.gov what kinds of information are you interested in?

   Topic questions:

Election summary race pages

Tested on feature: https://fec-feature-cms.app.cloud.gov/data/elections/house/AZ/01/2016/

Orienting scenario: You’re doing research on the 2016 Arizona district 1 election. And you’ve come to this page to learn more about this election. Prompt them to go back to these election summary pages if they navigate away.

1. Wendy Rogers was a candidate in this election. Where would you look to see the date of the last report filed by Wendy Rogers’ committee?
2. When was her committee supposed to file their pre-primary report?
3. Where would you go to find out about outside spending in this election?
4. The DCCC spent more than $2 million on independent expenditures. Can you tell me when they were required to report those expenditures?
5. Can you tell me if money was spent on electioneering communications for this election?
6. If you wanted to double check if a particular candidate is on the ballot, where would you look to find state contact information?
7. Let’s say you want to check when these candidates need to file all their reports. How would you do that?
8. Can you tell me which candidate is raising the most money in the current election for this district?
9. What is your overall impression of these pages?

Closing questions:

1. Any last thoughts about the features you looked at today?
2. Any other general feedback you’d like to provide, or any questions for me?
3. Not a question: Mention feedback button/github.

[JonellaCulmer]

Mural.ly board with user feedback: https://app.mural.ly/t/gsa6/m/gsa6/1522334628918/ae1f6c19a226ef079ff37ac9775380cb15a0c2b9

We've conducted the testing and came away with the following findings:

• All of the testers we spoke to were reporters and/or analysts and all visit FEC.gov often, sometimes daily. Most were interested in filing data, deadlines, how/where money is spent and raised including the money flow and IEs.
• Most looked to the Source reports column on the candidate financial totals table to find the last reports find by each candidate's committee. Some hoped to click on the candidate's name to get to the filings.
• When asked to find when a particular candidate was supposed to file, most relied on the Dates and deadlines card at the top of the page. While on that page a couple users had difficulty finding what they needed but new it was in that general direction.
• The "no data" table on electioneering communications does not accurately reflect the circumstances for previously held elections. This election tested was during the 2015-2016 cycle and did not correctly explain that there was no data since all expenditures should have been reported and processed by now.
• Users found all the data included in the table helpful and are also interested in additional visual depictions of our data.
• Users expressed a desire for alerts when a committee has filed a new report.
• Users are also interested in race-by-race comparisons.

Next steps:

• Draft new language to account for alternative situations for "missing" data on our no data messages
• Prioritize a redesign of the dates and deadlines section
• Continue with our plan to further enhance the election summary information by introducing the state-wide and nation-wide election comparisons
• Consider other data visualizations

New issue:

• Write new language for "no data" messages: https://github.com/fecgov/fec-cms/issues/1892