Research table filtering

[m-akinc]

Gather requirements and investigate possible API/implementation options. Output should be HLD. Might require pre-HLD conversations about cost-benefit of various options.

Options could include nimble table providing APIs or pushing all logic into client.

Considerations include the following (Fred will be working to nail down requirements):

• how filtering behaves with hierarchy
• searching all columns vs visible columns
• searching rendered data vs raw data
• providing APIs for menu-based filtering (all users with nameColumn="Abc Def") in addition to text search

[mollykreis]

Some key takeaways from looking into TanStack's filtering API:

• They have a boolean property of filterFromLeafRows that changes the way filtering works with hierarchy. The two possible modes are:
  • true: filter starting from the leaves of the data, which means parents of a matching row will be included even if they don't match the filter
  • false (default): filter starting from the top-level of hierarchy. If a parent doesn't match the filter, its descendants won't be included even if they would have matched the filter.
• The API supports both a global filter and column filters. Depending on what functionality we need to support, we could use one or the other or both.
  • TanStack has some out-of-the-box filter functions for columns, but I think there is a good chance we'll want to define our own function to be in more control of the match behavior.
  • Filter functions work off of the TanStack row, a column ID, and a filter. Nimble has a lot of flexibility with what it wants to do with that information (including using the column/cell-view to determine the formatted text associated with a given record).
• The type of the filter in TanStack is any, so nimble can define its own filter data structure if desired.

[rajsite]

API-wise, one approach we discussed was avoiding having the search logic in nimble table itself. Instead expose features from nimble to filter the set of rows and to calculate the presentation strings for rows. At a high-level, conceptually:

• setFilteredRecordIds
  • Just takes an array of rowids that are considered filtered (rows that are filtered in and still shown)
  • Preserves hierarchy (?) (i.e. making sure all the parents of a filtered rows are still present)
  • Preserves selection of rows that are filtered out (?) (unselect all potentially does not uncheck any rows you can't see
• getPresentationStringsforRecordIds
  • Takes an array of record ids and maybe a record of column ids
  • Waits for any configuration changes to have settled
  • Calculates the presentation string for columns (maybe only text columns (?))
  • Can maybe be calculated in a worker thread to avoid blocking the main thread calculating thousands of strings

Maybe we can also export algorithms (i.e. diacrtic search, etc) to help with searching across the strings.