edmullen commented 8 years ago

This issue relates to usability testing of the legal resources section conducted with internal power users from within the FEC on May 3, 2016.

Required activities:

[x] Create test materials
[x] Run test
[x] Post conclusions here
Test materials

Introduction:
Redesigning FEC site, focusing on legal resources
Asking people to try aspects of the site as we go, 30-45 min session
We're testing site, not you. You can't do anything wrong
Please think out loud, what you're seeing, what you're doing
Don't worry about hurting feelings, we need honest reactions
Ask any questions, I may not answer right away to learn how you respond
You are looking at a design prototype, not working code. Some things will be funky
Please tell us a bit about yourself and job
Task 1 - General browsing

Link: Homepage

Script:

You're conducting research to find existing legal guidance related to a topic, we'll use "filing" as the example. How would you do that?
[On Legal resources page] What is this page? What can you do here?
[Note the result page the user goes to] What is this page? What can you do here?
[If user did not go to All Sources > Results, ask them to] What is this page? What can you do here?
[When finished] Can you please go back to the main legal resources page?
Task 2 - Find an Advisory Opinion

Link: Homepage

Script:

Now you're conducting research and are only concerned with advisory opinions. How would you finding only AOs that relate to the topic of "filing"?
[Once the user gets to a page listing AO results]
Now select any AO and tell me what it's about based on what you see (not memory)
This AO cites a number of regs. How can you read those regs?
Task 3 - Find a Pending AO

Link: Homepage

Script:

cc @ethanheppner

edmullen commented 8 years ago

Test conducted May 3

Three users tested the the design prototype on Tuesday, May 3, 2016. I conducted and @ethanheppner assisted.

edmullen commented 8 years ago

Conclusions

Conclusions put together by @ethanheppner and me.

This audience immediately went to Legal Resources when asked to find legal guidance on “filings”.
Audience understood the general architecture of the Legal Resources landing page, it’s search box, and the subsequent results pages.
- Not all people limited the scope in the source menu. Those who did expected the reduced-scope results; those who didn’t understood the results groupings on the All sources results page.
In eRegs viewer of an individual subpart, having the navigation to other subparts at left appears to be helpful to those looking for more info. Proximity matters within the Regs because there’s topical structure to them.

AO result item structure is now sufficiently intuitive. Final opinion link, more hits, AO summary, snippet, and AO name link all understood.
Though initially dismissed by some, the FEC Record summary is seen useful in gaining a quick understanding of the topic of an AO after consideration.
Cross-referencing and linking cited Regs and AOs is seem as a big improvement that made users excited.
Users interpret “more hits” as other instances in the same AO where their search terms are relevant and might click on it to determine if the AO is pertinent to their research (Confirmation that changes made in XXXX are helping.)
To-Do: Calls to action on the AO landing page seem ill-suited to the activity they promote. The targets provide information, not actions you can take.
- Added note to #73
To-Do: Consider how to handle a user typing “Advisory Opinions” in the All sources search box. Maybe a typeahead to go to AO page.
- Added note to #78
To-Do: Make sure it’s clear that on the AO canonical page the Final Opinion button and link in the table refer to the same document - One user was confused by the “Final opinion” button being right above the “Final opinion” link-- they weren’t sure how to click.
- Created issue #97

Pending AOs were sufficiently prominent for people seeking them. People not seeking them were happy to see them when they happened across them.
Pending (AOR - Advisory Opinion Requests) being available in search results with closed AOs was also seen as positive.
To-Do: Learn about the numbering scheme and how/when an AORequest becomes an AO.
- Created issue #99
To-Do: Consider changing the label on the search field on AO results. “Terms” implies words and not AO numbers/names.
- Created issue #98

“Rulemaking projects” are a lot like Advisory Opinions: folders for various documents and stages of activities that are collected and lead to an outcome, regulations or changes. A rulemaking project usually focused on a topic or ruling and leads to changes in multiple sections of the code.
- Example
- These are more useful to some internal users than regs.
- AO canonical page approach may be usable as a rulemaking canonical page
Campaign guides (when to file etc.) would be a potential area for future plain language exploration of content that might be good to add to legal search