Data view: Search & Filtering

[jameskoster]

Part of https://github.com/WordPress/gutenberg/issues/55083

This ticket is a stub to include designs for search & filtering in tables. Considerations that could affect the design include:

• Scalability – there can be thousands of datapoints that a user might want to filter by. Performance, search, etc. all need to be factored in to the UI.
• Optimise for common use cases: It could be beneficial for some tables to present 'primary' filters to users by default.
• Advanced filter operators. These can be per-filter e.g: "Show posts in categories: recipes and reviews" vs. "Show posts in categories: recipes or reviews". Or even combined across filters e.g. "Show published posts and posts with no comments" vs. "Show published posts or posts with no comments". Clearly this can get quite complicated quite quickly, so should receive appropriate prominence.
• Saving filter configurations for reuse.
• Resetting; per-filter and globally.
• Shortcuts; create filter from column headers.
• Loading states while filters are enacted.

Iterations

Support pages, templates, and template parts pages.

DataViews component

• [ ] Top-level bar:
  • [x] Spacing issues in the top-level bar (see). #56151
  • [ ] Make sure it grows gracefully when there are many of them (see).
  • [x] @oandregal Create a more condensed view for filters in the list layout. Design. #56983
  • [ ] Disable filters when there's only one option. E.g. filtering pages by author is pointless on a site with only a single author.
  • [ ] Hide the combobox in search widget when there are more than X options. Probably use a selectlist.
• [x] Global search #54966
• [ ] Filter for ENUMERATION field. Target design. @oandregal
  • [x] Basic implementation based on SelectControl. #55270
  • [x] Users should be able to add/remove filters in the UI. The page should be able to define some default filters. Design. #55992
  • [x] Refactor the filter UX to target design #58569 @ntsekouras
  • [x] Create custom component based on DropdownMenu. Same UI in top-level & columns. #56110
  • [x] Allow resetting all filters at once. Design. #55955
  • [ ] Implement multiple selection. #56468 #59610 #59696
  • [x] Implement NOT IN operation. Design. #56479
  • [ ] Fix labels for buttons (see).
  • [ ] Support search & scrolling when there are many items.
  • [ ] Implement conditions "any" or "all". Design.
  • [ ] Logical operators: AND & OR. Needs design.
• [ ] Filter for DATE field. Starting design.
• [ ] Filter for TEXT field. Needs design.
• [x] Column filters. #55508
• [ ] Hook for DataViews consumers to easily filter their dataset.

Site Editor pages:

• "Pages":
  • [x] Global search (post title, post excerpt, post content). #54966
  • [x] Filter by author. #55270 #55455 #55723
  • [x] Filter by status. #55270 #55476 #55967
• "Templates": TBD.
• "Parts": TBD.
• "Patterns": TBD.

[jameskoster]

I shared these designs in https://github.com/WordPress/gutenberg/pull/55508 already, but adding them here for visibility too in case there's any urgent design feedback:

Filtering

https://github.com/WordPress/gutenberg/assets/846565/9aa0beff-0e6e-4e80-a206-6baaf37c0624

Loading & searching filter menus

https://github.com/WordPress/gutenberg/assets/846565/e272eb2f-3a05-455d-84d3-58e34827c241

[oandregal]

@jameskoster I've added a new section for tracking specific tasks.

Thoughts/Questions about some tasks in the backlog:

• For the pages page, what other filters can we implement? It sounds like we could allow filtering by date (the endpoint allows filtering by date before/after), and also a NOT IN filter for authors ("authors not in this list"). Is there a design for any of these?
• Improving the current IN filter: it sounds the DropdownMenu component to be used is not ready yet https://github.com/WordPress/gutenberg/issues/50459#issuecomment-1741012195 so this could be deferred a bit.
• Show/Hide filters: would users be able to hide the "default" filters for a page? For example, the "pages" page shows author and status filters by default – would the user be able to hide these? If the answer is not, I think we'd need to develop more filters before the show/hide feature makes sense.

I can start working on resetting all filters based on this design while we answer/unblock these questions.

[jameskoster]

For the pages page, what other filters can we implement?

For dates, it would be nice to offer preset options (e.g. 'this week', 'last week', 'this month', 'last month'), plus the ability to select a custom date range via datepicker. Would that be feasible? If so I'll work on a design.

Latest designs for the NOT IN operator make use of a 'Settings' flyout on the filter menu:

As you said, we may have to defer until the component is ready.

In terms of what else to filter by; discussion settings and template spring to mind, but they're lower priority. Template filtering would be multi select and require an 'any'/'all' setting.

would users be able to hide the "default" filters for a page?

Default filters should not be hide-able, but should be reset-able when a non-default value/setting is selected. This can be done locally via the "Reset" action in the menu (see screenshot above), or globally via the reset button in the main filter bar (the design you linked to).

[oandregal]

For dates, it would be nice to offer preset options (e.g. 'this week', 'last week', 'this month', 'last month'), plus the ability to select a custom date range via datepicker. Would that be feasible? If so I'll work on a design.

Yeah, absolutely. We need to figure out the preset => before/after conversion, but it seems doable.

Looking at the existing components, I see a DatePicker, a TimePicker and a DateTimePicker, but they do not work with ranges – the sooner we have a design, the better, so we can update them / create new ones as necessary.

I am also thinking that date presets need to be contextual to the dataset. There may be scenarios where shorter ranges are useful ('last hour' or 'today'), like for a store or a news site. Initially, we could offer some that cover a wide surface, and look at making them configurable per page/dataset if necessary.

[oandregal]

https://github.com/WordPress/gutenberg/pull/55270 implements the ability to show/hide filters.

[oandregal]

https://github.com/WordPress/gutenberg/pull/56110 starts implementing the new filter component.

[jameskoster]

Just a note to say that the datepicker design might take some time to get right, given all the permutations.

Until then, perhaps for date filtering we simply seek parity with wp-admin, IE filter by month. Hopefully that's simple to do with the current components?

[oandregal]

PR for implementing multi-selection in filter https://github.com/WordPress/gutenberg/pull/56468

(also updated a bit the issue description with latest progress)

[oandregal]

PR implementing NOT IN operator https://github.com/WordPress/gutenberg/pull/56479

[jameskoster]

Following on from the recent updates to the List layout, we need to reduce the width of the data view column. But before we can do that, we need a solution for displaying and managing filters in smaller areas. Filter labels can get quite long, and while wrapping is okay, it might not be the best solution:

An alternative approach could be to hide the filter dropdowns in smaller areas, and make filters manageable in the main filter menu:

Note how the Author filter remains visible in the menu even after it's been added.

In List layout, and other layouts on mobile devices this would work like so:

In the absence of the filter dropdowns, a small count ensures that it's easy to tell at a glance whether filters have been applied.

[oandregal]

First step to offer a more condense filter affordance at https://github.com/WordPress/gutenberg/pull/56983

[jameskoster]

Here's an alternative exploration for the general filter UX. Instead of shoe-horning everything into a DropdownMenu component with multiple flyouts, a single menu instance is used to add filters, and a popover instance contains the filter configuration.

This approach affords us more flexibility in terms of design which will be important when we add more operator options, and other types of filter such as text or date.

As demonstrated in the screenshot below, as soon as you elect to add a filter the popover is opened and focus transfers to the relevant element inside:

The operator is controlled via SelectControl at the top of the popover:

As a rough mockup, here's how this pattern could be applied to a title and date filter UI:

Narrow containers are still a bit of a conundrum, but there are options to explore there such as horizontal scrolling, or truncation:

[ciampo]

IMO this is a better UX. It also allows us to have better semantics, by keeping menu stuff in DropdownMenu, and having the filter UI in a separate popover.

We'll still probably have to go with a "custom" implementation for the popover, which will include:

• a listbox (ie. SelectControl) for picking the condition
• a custom data-picking widget (lika a combobox for text values, a date picker for date values, ...)
• potentially extra buttons (ie. to reset the filter)

Curious to hear @andrewhayward and @diegohaz 's thoughts too.

[andrewhayward]

Here's an alternative exploration for the general filter UX. Instead of shoe-horning everything into a DropdownMenu component with multiple flyouts, a single menu instance is used to add filters, and a popover instance contains the filter configuration.

This is a much improved direction, from a UX perspective; I was starting to get a bit worried that everything was going to be a menu!

We'll need to think a bit about how users interact with these, at a high level. For example, what happens if you + Add filter then immediately hit Escape? Does the filter collapse in place, having no effect? Or do we remove it, open the original menu again, and focus the appropriate menu item?

Similarly, what happens if the filter is already active and the user hits Escape? Does it undo any changes since you opened the filter popup? Or does it just collapse the popup, with the expectation that those changes are "live"?

The same applies for Reset too, but that's an even more explicit action. What does it reset the filter to?

[andrewhayward]

We'll still probably have to go with a "custom" implementation for the popover, which will include:

• a listbox (ie. SelectControl) for picking the condition
• a custom data-picking widget (lika a combobox for text values, a date picker for date values, ...)
• potentially extra buttons (ie. to reset the filter)

What @ciampo said seems about right to me; essentially a non-modal dialog positioned next to the relevant filter button.

We should be able to make a base filter implementation that handles the basics, and ensures a consistent experience; something like this brain-dump of pseudo-ish code...

function Filter ( props ) {
  const focusRef = useRef();
  const [ isExpanded, setIsExpanded ] = useState( false );
  const toggleIsExpanded = () => setIsExpanded(( expanded ) => ! expanded );
  const { title, activeCondition, values } = props;

  useEffect(() => {
    if ( isExpanded && focusRef.current ) {
      focusRef.current.focus();
    }
  }, [ isExpanded, focusRef ] );

  return (
    <PopoverButton isExpanded={ isExpanded } onClick={ toggleIsExpanded }>
      { `${ title } ${ activeCondition }: ${ values }` }
      { isExpanded && (
        <FilterPopover { ...props } ref={ focusRef } />
      ) }
    </PopoverButton>
  );
}

const FilterPopover = forwardRef(
  function FilterPopover ( { render, ...props }, ref ) {
    const { title, conditions, activeCondition } = props;
    return (
      <Popover>
        { title } <Select values={ conditions } selectedValue={ activeCondition } />
        { render( props, ref ) }
        <Button onClick={ ... }>Reset</Button>
      </Popover>
    );
  }
);

function TextFilter ( props ) {
  return (
    <Filter { ...props } render={ ( props, ref ) => (
      <TextInput { ...props } ref={ ref } />
    ) } />
  );
}

function ListFilter ( props ) {
  return (
    <Filter { ...props } render={ ( props, ref ) => (
      <ComboBox { ...props } ref={ ref } />
    ) } />
  );
}

function DateFilter ( props ) {
  return (
    <Filter { ...props } render={ ( props, ref ) => (
      <DatePicker { ...props } ref={ ref } />
    ) } />
  );
}

We'll also have think about how translation works here; for example, ${ title } ${ condition } (<Title> <starts with>, <Category> <is>, etc) is fine in English, but might not be in other languages.

[jameskoster]

For example, what happens if you + Add filter then immediately hit Escape? Does the filter collapse in place, having no effect? Or do we remove it, open the original menu again, and focus the appropriate menu item?

Good question. On the surface the latter seems more helpful, but if you're someone who wants to add filters before configuring them it might be unexpected/frustrating?

Perhaps it's best (and probably simplest) to start with consistency. So regardless of whether you just added the filter, or added it previously an re-opened the configuration popover, pressing escape would close the popover and transfer focus to the trigger.

As for saving, the changes would occur "live", so in effect there's nothing to discard or apply on close.

[SaxonF]

+1 on this approach @jameskoster . It also aligns better with the idea of primary filters that are always visible for quick access but may be empty. We could still condense into a single dropdown on smaller containers.

[ntsekouras]

I've started working on the redesign and hopefully very soon will have a PR.

[jameskoster]

For the filter buttons / chips we might try something like:

This would produce the following:

[oandregal]

https://github.com/WordPress/gutenberg/pull/59610 implements multi-selection for filters.

[oandregal]

@jameskoster is there any design for filters of type "open text"? This is, filters that don't have a preselection of values. At the moment, this is done via the global search, though we may want to have a dedicated filter for that with support for operators such as "Starts with...", "Ends with...", "Contains..." (the current global search).

[jameskoster]

Yes!

It would be good to get thoughts on the operators here. We could potentially start simple ("Contains" / "Does not contain") and expand from there. What do you think?

[oandregal]

What's the relation between this new filter and the "global search"? The global search is quite opaque in that the user doesn't know the fields or operators that it uses. I wonder if both should be connected.

[jameskoster]

Good question. The global search currently looks for matches in title and content, is that right? That seems like a fairly natural default behavior to me, for now at least. Perhaps in the future it can support filtering syntax, and reflect filters added manually, a bit like the search on github issues.

That would need some more design exploration though, and I'd welcome ideas!

[oandregal]

https://github.com/WordPress/gutenberg/pull/59953 adds support for AND operators — there's no current use case in pages, templates, parts, and patterns though it's needed in future post types (post, media).