Implement and test site-wide search prototype

[noahmanger]

This is an exploratory issue to help us better understand the user needs (and potential ways to address them) around a site-wide search. To do this, we will use a combination of combing through external resources and doing lightweight prototyping using the out-of-the-box DigitalGov search tools.

Questions to consider

• How do we handle search results from the new and legacy sites?
• Is there a typeahead component to allow users to jump directly to certain pages?
• How do users expect results to be presented? What sorts of things do they expect to search through?

Completion criteria

• [x] Examine search logs from current fec.gov for trends and patterns
• [x] Gather and synthesize external research on site-wide search patterns and best practices
• [x] Create a basic implementation of the DigitalGov search tool (perhaps with some version of a typeahead)
• [x] Test the prototype with users
• [x] Consolidate findings and make recommendations for next steps

[onezerojeremy]

for reference, here's the search analysis we did when we were spinning up the legal research: https://docs.google.com/document/d/1CBxO99tKjg2w2D4dMIxx_1KYEnbptK3VGqi8rJsDbt8/edit#heading=h.eezo9x7qpz6w

[patphongs]

Here's notes from our UX mind-meld:

Questions to answer:

• Can the search crawl both old and new sites simultaneously?
• How do we display content for both old and new sites? Should there be a difference
• How does the search crawl?
• What is the search capability? To only crawl static wagtail content? Or will it be more global?
• Does it crawl PDF files?
• Should research the user need, what are users really trying to search if this is implemented?

[patphongs]

Here's an issue on exploring and prototyping the digitalgov search option: #850

[noahmanger]

Moving this issue to next sprint.

[noahmanger]

I spent some time today looking into various technical approaches and have some findings:

DigitalGov search

First, the DigitalGov search API doesn't exactly come for free. The only way to use it is to either 1) use their i14y indexing or 2) purchase a Bing key (Google's Site Search is the other option, but apparently it's being discontinued).

The Bing search key costs several thousand dollars and the i14y API requires a process of manually pushing content updates to their API to tell it to index pages. While they have plugins for Wordpress and Drupal, we'd have to roll our own methods. This is an option, but it's more involved.

If we're not using their search results API, it means using the default search results display. This could be ok, but we're running into issues with the fact that the default indexes of candidate, committee, and legal resource pages are incomplete.

I was hoping we'd be able to address that by using our API directly, but that isn't possible if we're using the DigitalGov search page.

The other way to address that, however, is to do something that we should do anyway, which is to improve the search indexing of all the pages.

Wagtail search

The other option that I spent a few minutes looking into is to take advantage of Wagtail's built-in search, which you can set up to use ElasticSearch. The benefits here is we'd be able to customize the results page to also pull in results from the API. The major downside is that we're effectively rolling our own site search which is very tough to do well. But hey, it's an option.

[onezerojeremy]

Another update on this: Had a discussion on this at design pairing. list of things that happened:

• talked through some of the thinking
• demo'ed mock-ups (see below) of a potential federated auto-suggest that allows users to have a single simple query return results of multiple types in the type-ahead.
• talked about some of the technical difficulties of doing this.
• action: going to investigate digital gov search's nooks and crannies for interface elements we might be able to tweak - @onezerojeremy
• action: investigate using the type-ahead as a way to link to the digitalgov search as a simple integration (@onezerojeremy to create ixd, @noahmanger to investigate technical feasibility)

There may have been some other takeaways and actions, please add what I missed!

Mock-up: federated type-ahead showing search for "ele" returning results across multiple content types

Mock-up: federated type-ahead showing search for "electronic" returning results across different content categories

Faceted search showing one option on top level facets or categories that align with content sections

Faceted search showing how sub-facets might appear for main facets

Faceted search showing a different option for facets that align with things most commonly searched

[noahmanger]

After tinkering with the DigitalGov settings and experimenting with the typeahead, Jeremy and I chatted and we think we can make this work for our needs.

We'll discuss in sprint planning what we want our next steps to be, and all agree on if we actually want to pull the trigger and do this. In the event that we do, I went ahead and made fresh issues for the next design steps on this front in:

• https://github.com/18F/fec-style/issues/656
• https://github.com/18F/fec-cms/issues/935

There's one last piece which is to make some changes on our end (and then using the Bing webmaster tools) to improve search indexing of candidate and committee pages, but still working through the details there.