Research how users navigate to audit report search

[PaulClark2]

Figure out how users navigate to audit report search (where it falls in info architecture and navigation)

• [x] Design necessary navigation modules and where they're used/placed for getting users to audit report search
• [x] Mock up placement and styling
• [x] Move to design in fec-cms 1311

[JonellaCulmer]

Working the @llienfec to identify individuals for user interviews. The audit team is a major end-user for this information on FEC.gov and will help point us in the right direction.

These interviews are slated to take place next week, but scheduling is still TBD.

[JonellaCulmer]

https://github.com/18F/fec-testing/blob/master/Testing-Scripts/101017_NavigationtoAuditSearch_TestScript.md

October 10, 2017 Test Script

Quick connection links:

Notes spreadsheet: https://docs.google.com/spreadsheets/d/1Z3T45hb1WAhD1zRL_fLXZlfQ9ApenTvgYE0OM0eLhaE/edit#gid=313597352

Features tested this week:

Audit Reports Navigating to audit search New audit designs Open-ended feedback

Introduction script:

Hi, I'm ___, and I'm working on the Federal Election Commission website redesign, and we'd like to get your feedback on some new search features we're adding. Before we start, I want to make it clear that we're testing the website, not you. You can't do anything wrong here - all answers are the right answers. Although it may seem a little weird, I'm going to ask you to think out loud as much as possible. Describe what you're looking at, what you're trying to do, and what you're thinking. This will be a huge help to us. Finally, please don't worry that you're going to hurt our feelings. We're doing this to improve the site, so we need to hear your honest reactions. Sound good? [wait for user confirmation; clarify things if user asks questions] (credit: Krug)

Introduction questions:

Before we take a look at the website, I’d first like to ask a few things: How often do you use the audit search on FEC.gov as a part of your work? What prompts you to conduct an audit search on FEC.gov? What kind of information are you hoping to gather when you conduct an audit search?

Audit search questions

Task 1: FEC.gov: Can you walk me through the steps you typically take to find what you were looking for when conducting an audit search? (Probe to find out whether search is usually conducted to find a specific audit, or audits that reference specific findings or issues?) Task 2: Classic: Can you walk me through the steps you would typically take to find what you were looking for when conducting an audit search? (Probe to find out whether search is usually conducted to find a specific audit, or audits that reference specific findings or issues?) Do you have any general impressions of the current audit search?

New audit design questions

InVision prototype Show tester mock-ups of audit search page Before we take a closer look at this page, what is your general impression? Task 3: Looking at this new design, how would you begin your search? Task 4: What would you expect to happen if you clicked on American Financial Services Association PAC there in the search results? Move on to audit report landing page. Task 5: Go to the Gary Johnson - 2012 audit report. Task 6: You’ve found the audit you’re looking for, can you select the final report? What are your general impressions of this audit report page? Is there anything you expected to see, but did not?

General questions

Those are all the questions I had! Are there any questions you wish I asked you but didn’t? Do you have any questions for me?

[JonellaCulmer]

Mural board with notes from testing: https://app.mural.ly/t/fec6677/m/fec6677/1507906314492/6f8cf6077eeb2cc80e7c8d6c670da772c2cdcced

Findings Summary

Usage Findings:

• Participants use the search as necessary. When conducting an audit review the ARSS system is used, but can go a few weeks without using it.
• Primarily searching for precedent details and specific wording of previous audits.
• Use committee type summary pages when a specific audit is known, or use ARSS when conducting a more topical search.
• Need to see how past audits were worded and organized.

Navigational Findings:

• Have trouble finding the audit search information on the Enforcement page.

• Have trouble find the ARSS on the Audit Reports page.

• Some users have not used the new site for search yet, still use classic.fec.gov

• Primarily use tiers to conduct audit search in ARSS. Sometimes filter by date and committee type.

• There isn't such a long journey to the audit search information on classic as there currently is on fec.gov.

Current Audit Search Findings:

• Think it's easy to navigate and find relevant audits.

New Design Findings:

• General consensus that the search looked clear and had the same filters/features to conduct a search as they normally would.
• Majority use the findings and issue categories to search
• Nearly all participants thought that once they clicked on a particular audit they would be directed to a complete list of information and documentation regarding that audit.
• All participants easily found the relevant audit in the list of search results.
• Most found the Final Audit Report button easily. But few went straight to the list of complete documents hoping to find it there.
• Participants generally like the layout of the Audit Report landing page including the organization and inclusion of findings.
• Reiterated desire for more filtering by keyword.

[JonellaCulmer]

Two proposed solutions to improve Audit search navigation. Both are search functions.

A huge pain point in the audit search navigation was getting from the Enforcement page to the Audit Search page. There are currently two links that are not easily seen and are hindering users from accessing the Audit Report Search System. At this point, my recommendation would be to include a search feature on both the Enforcement landing page, as well as the Audit Reports page.

Option 1

The first solution is a search box that will allow users to search for the Committee Name, or Authorized Candidate Name or ID of either. This will be helpful for getting users straight to the Audit Report search system. There they can include additional filters if they wish. The candidate/committee search function is helpful to those who know the specific audit or committee they're looking for and not for a more topical search.

Option 2

The second solution would be a dropdown search box that will allows users to select from the categories listed in the Tier 1. Most of the users tested tend to use the Tiered categories more than any other search option on the Audit Report Search System. This will benefit users who are conducting more of a topical search instead of searching for a specific audit.

This solution may also allow for the use of both tiers or both the main category and subcategory.

[jenniferthibault]

Additional background

Since auditors at the FEC could not speak to how people outside the FEC use audit reports, when I started redesigning the audit search and canonical case landing page, I reached out to Christian from the Press Office for preliminary info on what members of the public look for when contacting the FEC about audit. He provided the following:

The Press Office receives questions on audits infrequently, but when members of the public ask about audits, here are some of the common audit questions:

1. Has a particular committee been audited?
2. How does the Commission determine which committees/candidates to audit?
3. When will new audit reports be released?
4. I noticed several documents and memorandums in connection with an audit. What are the differences?
5. I read a recent open meeting news release containing an interim audit report. How come I don't see that on the audit page?
   • Answer: Most agenda items discussed at an open meeting are made available to the public through our website. A committee has up to 30 days to provide responses to the findings in an Interim Audit Report. At a later stage and once the Commission approves the Final Audit Report,a copy will be sent to the committee. Once this report is received by the committee, it will then be placed on the Commission’s audit page along with Committee responses and other documents related to the audit.
6. I didn't see any enforcement action in the Final Audit Report. Will the Commission assess a penalty? If so, how can I find out?
   • Answer: If any findings are referred to the Commission’s Office of General Counsel or to the Office of Alternative Dispute Resolution (ADR) for further enforcement, the Commission may open a separate enforcement case. The matter remains confidential until the enforcement case has been closed. You may track closed enforcement cases through the Commission's Weekly Digest that is published on Friday afternoons.

Takeaways

The biggest fundamental difference between internal and external users follows a known pattern on the site: Internal users use the site for precedent and data checking, external users search by which candidate or committee is involved. We need to serve both, but I think what will help us make decisions about the direction is that we've always focused on serving the needs of external users of the site first.

Feedback on designs above

• The recommendation to include a search feature on both the Enforcement landing page, as well as the Audit Reports page seems right on to me—in the interaction pattern established (and proven to be recognized through the research!) of the new site, when a search exists on new fec.gov, we don't just link to it, we embed it so that people can start their search where it's useful to them.
  • My "yes AND" to that, is I think in order to make the Enforcement page more easily scannable to people, I think we may want to ask for content design help. ( 👋 @jameshupp @llienfec @AmyKort ) During last week's testing, @JonellaCulmer watched people skim down the Enforcement page and not see audits—if people were looking to the left-hand menu, and deducing what they were looking for wasn't there, how can we improve the content displayed on the menu to show those keywords? I remember running into this problem with people who are searching for MURs as well. Since the MUR section is farther down the page and people at the top didn't see the word "MUR" in the menu or above the fold, they assumed they were in the wrong place. (Editor's note, I'd like to come back and link these ideas to research finding posts) Some first thoughts on approaches:
  • Should we restructure the section titles to group them by topic instead of by theme?
  • Should we rewrite the section titles to include the programs in the name?
  • Another approach our content geniuses can help identify?
• A slight adjustment to the module: As a pattern of behavior, when presented with the embedded search functionality on a page, we've found that folks need a way to get to the search results page without entering anything first. You can see an example of this in the "Advanced search" link button for AO's and a fix for that on staging for MURs
  • 
• If we include the "Advanced search" button that gets people to the full functionality, I think it's safe to lead with just the candidate/committee embedded search field until a later date when we can support keyword search for audits. We could do both the Candidate/committee and Category search using the OR search above, but I wonder if it's better to keep it to just one to begin with.

---

I know that's a lot to take in, but wanted to get it all out before I run off to conferencing! Looking forward to your thoughts @JonellaCulmer ?

[JonellaCulmer]

Thanks for that background info, @jenniferthibault! That's very helpful to get the external users perspective in this too.

I agree that the Enforcement page organization may need to change, including the possibility for content to change the groupings of information by topic instead of theme. Although that may leave very little content in some sections. Which makes me think of a Regulations-type page instead with links/buttons to those specific sections.

I think that Advisory opinion search with advanced search you mentioned would work great on the Audit reports page and give users, internal or external, both search options by candidate/committee or ID and by category. And we won't have to prioritize helping external users first in this case.

For the Enforcement page, we may have an opportunity to move up the MUR search as well. We could explore using Campaign Finance Data landing page and include a search for Audits and MURs. If we moved up the search for both, we can keep the rest of the content mostly as is. See whiteboard doodle below that also incorporates the Advisory opinion type search.

[JonellaCulmer]

Issues that may support moving the MUR search up as well: https://github.com/18F/FEC/issues/4583, https://github.com/18F/FEC/issues/4472

[AmyKort]

Research is complete. We'll move on to implementation.