Redesign Search

[JakeHartnell]

Our search UI needs work. In short, faceted search is confusing, ugly, and requires more work on behalf of the user. I believe we can have all the power of the faceted search with less of the confusion. While visual search is cool, from a usability standpoint it acts quite differently from how people are used to interacting with search boxes (i.e. just typing something in).

One problem is that if you just type something into the box without assigning it the right category, you are prone to getting no results (even when there are some). For example, typing my username into the box without selecting user gives me zero results:

This is annoying (an extra step that doesn't seem necessary [it also sucks on mobile]). I can't think of another platform where search works like this, and I think there is a good reason for that: it's currently poorly designed.

In this thread I would like to open a discussion and get some alternatives proposed for how we can improve this.

[aron]

Great initiative. I don't think faceted search is necessarily confusing, merely our current implementation.

I'd love to see a simple catch all search box that searches across most of the key fields (user, quote, text, etc). Plus an "advanced" mode that allows results to be filtered by individual criteria.

A super quick sketch to illustrate:

We'd have to see if the simple search addressed most cases, but I think by moving the "pill" facets into a separate component that has a more step by step interface (e.g. step one: add filter, step two: select category, step three: input value) it makes the whole thing much clearer, and also fixes the fiddly small hit areas of the current interface.

[csillag]

We could introduce a new facet, called "any", which would search everywhere, and join all the results into one list. This could be the new default.

That way, users "just typing something in" would get all the results they expect, but we could still have the flexibility and power of faceted search.

[gergely-ujvari]

Are we talking about the stream page or the page search feature? Currently they both use the same visualsearch bar, and I totally agree with the search-all/search-any feature for the page-search, do we really want to have our backend to search in all major fields?

[aron]

@gergely-ujvari I think functionally they should behave the same way in both cases for consistency.

I think the same approach should ideally be taken when searching across all annotations. Is there a user related reason why we wouldn't want this? Or is it purely implementation?

If implementation is costly then we could revise the focus of the stream page to meet somewhere in the middle.

[gergely-ujvari]

@aron: purely perfomance. The implementation won't be hard, I'm a bit worried about the performance cost of an any search.

[csillag]

I think I think @gergely-ujvari has performance concerns for the any/all search running on the full stream. We will have to do some measurements to see how big is the overload, compared to, say, text search only.

[aron]

We will have to do some measurements to see how big is the overload, compared to, say, text search only.

@csillag @gergely-ujvari of course, that sounds sensible. No point having a feature if it kills our servers or takes minutes to return data :)

[tilgovi]

I am not an all concerned about the performance. It's the kind of problem that one can scale by throwing hardware at it. And that's easy when your hardware is virtual and your search software scales horizontally.

I support removing the visual search widget in favor of a basic search of common fields and then adding an advanced search in the future, as a separate iteration.

[csillag]

If we can add a basic search facet (which searches everywhere), and make it the default, would any of the problems remain?

I fail to see a reason for removing what we have, and works almost perfectly. (Except missing the "search everywhere" feature, which we can easily add.)

[tilgovi]

I fail to see a reason for removing what we have, and works almost perfectly.

First, It's confusing and ugly. These were the words of @RawKStar77 but I agree. Particularly the "scope" facet has all the confusion of our view settings (user asks: what does "screen" mean? etc)

Second, it's 85K of page load bloat. That's underscore + backbone + visualsearch.css. Measurements taken after minification.

Lastly, it's a usability concern. Faceted search is not common, whereas advanced search panels are somewhat understood and can be hidden more easily.

Simple is beautiful.

[gergely-ujvari]

I think I can prepare a test branch for this during the next spring. It'll have only one search bar, no visual search, and by default, search-all relevant fields. These fields should be:

• Quote (if root annotation)
• Text
• URI
• User
• Tags

Scope currently won't be handled. Just one question, shouldn't we keep the since, and hits/results field somehow (only relevant for the stream) of course they'll have a default value but these two are different from the rest of the facets

[dwhly]

Good topic for the design meeting.

[dwhly]

@gergely-ujvari said:

I've prepared an alpha demo version of a possible simple search bar instead of visualsearch. I want you to try this out to see if we can proceed this way or not.

Just before giving the url, some necessary warnings:

• Stream is not working yet, so please, just use page-search
• It is not styled, it is yes, ugly, not responsive, but we can easily do this after we decided to have this feature or not
• Since it is alpha, errors can occur
• Code is really pre-alpha, just a proof of concept

Features:

• If you click on the magnifying glass icon, instead of VS, you get a simple text input field
• You can write your search expressions here
• Space acts currently as a separator character between expressions So if you type the following string into it and hit enter: 'ujvari tests searchbar' then it'll check the 'quote', 'user', 'text', and 'tags' fields if any of them matches into 'ujvari' or 'tests' or 'searchbar'

Mixed results are valid too, so i.e. if the username is ujvari, you have the word 'tests' inside the annotation body and it has a tag called 'searchbar' then this annotation is a search result So the rule is, all expression must match in a field searched.

Besides this you can narrow down the field by putting specific prefixes before a search expression. Like: "u:ujvari" means that "ujvari" has to be username of the annotation "t:label" means that "label" has to be a tag added for the annotation "b:body" means that "body" has to be in the annotation's body (text) "q:verse" means that "verse" has to be in the annotation's quote

Expressions can come in any order and you can mix between 'any' expressions and field-specific expressions. I.e. the search expression "u:ujvari tests searchbar q:annotate" is legal and works.

Dokku link: https://simplesearch.dokku.hypothes.is/

Last words: I've made many part of the search processing configurable, as preparing for the future, and even if this feature is not needed, some part of it can go as a refactor pr in the future.

[dwhly]

From the design meeting today:

@aron will work w/ @gergely-ujvari to style the search box and we'll ship this in the current sprint w/ power user facets if possible.

[dwhly]

@bigbluehat said:

"Also, if we go this route, we should also support the full names for facets, not just their abbreviations (does "t" stand for tag or text?)."

Extrapolating that a bit might look like this: user:ujvari u:ujvari (since this could also be "uri", do we want to assert that "u" == "user"? maybe not?) tag:label text (or body?):string q:verse quote:verse uri:domain.tld since:Xminutes (specify an arbitrary number of minutes/hours/days?) results:30 (specify an arbitrary number of results)

putting quotes around something might allow spaces to be in those search strings?
quote:"to be or not to be"

combining facets would be supported: user:dwhly uri:nytimes

[tilgovi]

May I suggest we only use the full names for facets?

[dwhly]

+1 happy.

[gergely-ujvari]

WIP PR for this issue is #1326

[gergely-ujvari]

New dokku instance: https://simplesearch2.dokku.hypothes.is/ (BTW, can I force push a rebased branch into dokku? Then I didn't have to change the instances.)

What's new:

• rebased to more up-to-date master branch
• support quotes, so now you can search for "to be or not to be" (or even "I'd rather not 'say' this")
• full names for facets:
  • quote:"The web annotated"
  • result:5 (limit the search to 5 hits)
  • since:5min (also supports since:30sec, since:2hour, since:6day, since:2week, since:1month, since:4year) (I've a grammatical question here, whether to play with the plural forms or not?)
  • tag:useful
  • text:Detail
  • uri:http://www.example.page.com (will come to play for the stream page)
  • user:ujvari
• errors are also expected in this phase

The stream page is not working right now, will work on that.

[gergely-ujvari]

Latest version here: https://simplesearch3.dokku.hypothes.is/ (Stream page also working)

[tilgovi]

Simple search is merged. Hurray!