Define data grid component internal API

[mj-hof]

🚧 Request For Comments! 🚧

The following proposal is up for discussion and will change. The proposed APIs are not complete. Please add your input as comments.

• Why?
• Description
• Implementation
  • 1. TableApiLayer
  • 2. TableLogicLayer
  • 3. TableRenderLayer
• Design

Why?

The aim is to have an extendable data grid with separated concerns and avoid too much intertwined code. So we define in this ticket the basic internal architecture for the future OnyxDataGrid.

Description

We begin by defining what table state is: Everything in our data grid that can change over time. So what can change?

• ✅ order (sorting, drag n'drop)
• ✅ existence (filtering, deletion, creation, pagination)
• ✅ input values (:tableData="tableData")
• ✅ input metadata ("highlighted")
• ✅ changed values ("edits")
• ✅ internal metadata ("checked")

What is not considered state?

- ❌ ongoing user interactions, e.g. [composition](https://developer.mozilla.org/en-US/docs/Web/API/CompositionEvent) or when the user is dragging an element, but hasn't dropped it yet

=> state is the sum of these data

or

"State is a function of data and context"

or

$State = f(Data, Context)$

Implementation

Technically, we will achieve this using three layers:

1. TableApiLayer => provides a neat API for the users and hides the internal logic
2. TableLogicLayer => holds all the logic and creates an immutable state object (type of RenderTable) representing the table (our "State Function")
3. TableRenderLayer => uses the state object to render the table.

1. TableApiLayer

TBD - not relevant for this PoC

type OnyxDataGridProps<TableData> = {
  columnDefs: {
    key: keyof TableData;
    /**
     * Provide your custom render Component.
     * The value of the cell will be per `modelValue` prop.
     */
    customRenderer: Component<{ modelValue: TableData[keyof TableData] }>;
  }

};

2. TableLogicLayer

ℹ️ For the following examples and definitions, we assume a table is rendered out of the following, very simple API:

type SimpleHeader = {
  label: string;
}

type SimpleRow = {
  value: string;
}

type SimpleTableAPI = {
  headers: SimpleHeader[];
  rows: SimpleRow[][];
}

In this layer, we take the provided user table data and transform and extend it. Every feature acts as a transform with the following signature: function enrichTable(userTableData: SimpleTableAPI): SimpleTableAPI.

Ideally, we implement our features with the same API that we provide for others ("Eat your own dog food").

We can optimize the transformation process by taking a look at the aspects of a table feature:

• adding/changing/removing row data
• adding/changing/removing column definitions
• change how cells are rendered
• change how headers are rendered

adding/changing/removing row data

We can imagine features which mutate row data as a sequence of array mutations: E.g. :

1. Add user defined rows (which were added through clicking "add row")
2. Add virtual rows (load more...)
3. Filter user deleted rows
4. Filter user filtered rows (when user has defined a filter)
5. Mark user selected rows
6. Apply user sorting.

This can be abstracted into the following steps:

1. Array mutations before
2. Array mutation (map)
   1. Map entries before
   2. Map entries
   3. Map entries after
3. Array mutations after

By grouping all mappings together, we can optimize the performance.

Adding/changing/removing column definitions

There is no need for optimizations here, as we can assume the rendered columns count to be a low. Therefore, we can just provide a simple mapping function.

Change how cells/headers/rows are rendered

For rendering, we make use of Vue's FunctionalComponents. These are always stateless, which is what we want for performance and philosophy reasons. The component must always take a Header component as a prop and wrap/use it.

E.g. a header could be extended like this:

const WithSortIcon: FunctionalComponent = 
  ({ Header, data }) => 
    h('div', [
      h(SortIcon, { sortDirection: data.sortDirection, onClick: data.onSort }),
      h(Header)
    ]);

E.g. a row would work like this

import OnyxNumberCell from "./table-fields/OnyxNumberCell.vue";

const NumberCell: FunctionalComponent = 
  ({ Cell, data }) => 
    h(OnyxNumberCell, { value: data.value, editable: data.editable });

The same approach can be used for rows and cells.

Based on the above, I propose the following API:

/**
 * keeps track of all changes by every feature for a single row
 * 
 * @example type {
 *   filtering?: Metadata;
 *   deletion?: Metadata;
 *   creation?: Metadata;
 *   editing?: Metadata;
 * }
 * 
 * @example content {
 *   filtering: { hidden: true };
 *   addition: { hidden: false };
 *   editing: { edits: { fruit: "apple" }, hasChanges: { fruit: true } };
 * }
 * 
 */
type Context = Record<TableFeature['name'], Metadata | undefined>;

type EntryState<Key, TableData> = {
  id: Key;
  data: TableData;
  context: Context;
}

type State = EntryState[];

type Order = {
  /**
   * @default 'after'
   */
  order?: number | 'before' | 'after';
}

type TableFeature<TableData, Metadata> = {
  name: string;

  /**
   * Allows to add context to table rows.
   */
  mapping?: {
    /**
     * The returned metadata is added (if not undefined) to the context object with the table feature name as key.
     */
    func: ({id: Key, entry: Readonly<Entry>, context: Context}) => (Metadata<TableData> | undefined);
  } & Order;

  /**
   * Allows modifying the table state as a whole.
   */
  mutation?: {
    func: (state: State) => State;
  } & Order;

  /**
   * Allows the modification of the header columns before render.
   */
  modifyColumns?: {
    func: (cols: RenderColumn[], colDef: ColumnDefinition) => RenderColumn[];
  } & Order;

  /**
   * Define how a specific metadata entry of a row is mapped.
   * @default Depends on the metadata key. Usually uses the first non-nullable entry.
   */
  reducers?: Record<keyof Metadata, (context: Context) => RenderRow>;
};

/**
 * Metadata that is passed to the row as render information.
 * @example 
 * ```ts
 * { 
 *   hidden?: boolean;
 *   virtual?: boolean;
 *   created?: boolean;
 *   edit?: TValue;
 * }
 * ```
 */
type Metadata<TValue> = Record<string, any>;

General table data pipeline:

1. user mutation before
2. built-in mutation splice-additions
3. built-in mutation splice-virtuals
4. built-in mutation mapping
   1. user mapping before
   2. built-in mapping deleted (e.g. user clicked delete)
   3. built-in mapping filtered (e.g. built-in filter func)
   4. built-in mapping user-filtered (custom user filter func (entry: Entry) => boolean, only called if not already marked deleted or filtered)
   5. user mapping
   6. user mapping after
5. user mutation
6. built-in mutation filter
7. built-in mutation sort
8. user mutation after
9. built-in mutation: map to RenderRow
10. Generates SimpleTableAPI state and passes it to the TableRenderLayer

3. TableRenderLayer

A minimal API that renders an HTML compliant table with only permitted content elements:

Permitted content

In this order:

• an optional element,
• zero or more elements,
• an optional element,
• either one of the following:
  • zero or more elements
  • one or more elements
• an optional element

The TableRenderLayer must be stateless. Its main purpose is to create an HTML compliant table structure. The table should be rendered according to the following rules:

• columns describes the horizontal layout of the table, every entry MUST result in a column and <th>element
• rows describe the vertical layout of the table, every entry MUST result in a row and a <tr> element
• so there should not be any "unused" entries in either column or row
• extensions should only follow table concepts that are supported/specified by the HTML spec
• the entry keys of a row must match the specified columns or colgroups
• the content of the table data cells (<td>...</td>) in a row can be any content

/** API */

/**
 * Props of the TableRenderLayer
 */
type RenderTable<TableEntry extends object> = {
  idName: keyof TableEntry;
  columns: RenderColumn<TableEntry>[];
  rows: RenderRow<TableEntry>[];
}

type RenderColumn<Key extends keyof TableEntry | symbol> = {
  /**
   * Key of the column - usually a key of the tabledata.
   * But can also be used for custom columns.
   */
  key: Key;
  /**
   * Attributes and data that is provided to the component using `v-bind`.
   */
  cellAttrs?: TdHTMLAttributes;
  /**
   * The component that renders the header content.
   */
  header: FunctionalComponent;
}

type RenderCell<Key extends keyof TableEntry, CellData extends TableEntry[Key]> = {
  /**
   * Key of the column - usually a key of the tabledata.
   * But can also be used for custom columns.
   */
  key: Key;
  /**
   * Data that is provided to the component using via the `metadata` prop
   */
  metadata?: object;
  /**
   * table data that is provided to the component using via the `metadata` prop
   */
  data: CellData;
  /**
   * The component that renders the actual cell content.
   */
  cell: FunctionalComponent;
}

type RenderRow<IdKey extends keyof TableEntry, RowId extends TableEntry[IdKey]> = {
  /**
   * Unique id of the row.
   */
  id: RowId;
  /**
   * Data that is provided to the row component using via the `metadata` prop
   */
  metadata?: object;
  entries: RenderCell[];
  /**
   * can be used to add attributes, including class, style and even event listeners
   */
  rowAttrs: HTMLAttributes;
}

Design

Figma

[JoCa96]

RFC: @SchwarzIT/onyx-dev @SchwarzIT/onyx

[JoCa96]

Thanks @BoppLi for challenging the RFC: bullet points of the challenges:

1. column config? e.g. reordering the columns e.g. column resizing
2. lazy rendering? -> virtual scrolling support
3. slots for cells?
4. pinned rows / columns?
5. are we sure that re-renderings of the pipeline will not accidentally "kill" e.g. > unsaved changes?
6. interference of e.g. "active page" in pagination vs filterValue changes
7. how does saving changes affect the pipeline? the "temporary state" would be dissolved > and the prop data would change, how to make sure those don't collide? -> who owns which state?
8. any forseeable issues with keyboard navigation?
9. how are events bubbled up? we had immense problems in CoreUi with that where we had to > pass them on manually (edit mode), and with tooltips, it did not even work at all
10. style definitions e.g. on columns, like "monospace", alignment, truncation, ...

Solutions

1. Column configuration is provided similar to how the API is currently in CoreUI. The user passes the config in through the TableApiLayer.
   • Reordering the columns should be doable through the TableFeatureAPI:
   • adding the drag element can be done through modifyColumns
   • the drag element can listen for dragstart
   • modifyColumns can also be used to add virtual columns that can be styled and used as drop targets
   • For resizing columns modifyColumns can also be used to add virtual columns that can be styled as resize borders and that have event listeners
2. Lazy rendering is such a foundational feature that it would also be fine to implement this in the Render layer as a base feature. But it would be possible to implement this through the TableFeatureAPI:
   • We require the table row height to be static and defined beforehand.
   • We can extend the TableFeatureAPI and add customRowRender option that allows us to control the row rendering. The renderer can then render all rows initially empty with the defined height. Then an intersection observer can be used to figure out which rows should actually render their cells.
3. The TableRenderLayer expects a component-ish to be provided for every cell content. There we can easily call a user-provided Component with a predefined API.
4. With the TableFeatureAPI we can reorder pinned rows and columns and mark the pinned entries. We add a reducers that adds a class to all pinned rows/column cells.
5. Should not happen if we use the vue key attribute correctly.

6. See 7.

7. The user needs to bind the v-models correctly and consider our emits, than that should not make any issues. State is owned by the TableLogicLayer. But we could consider an extension similar to our useManagedState approach, where we optional pass control over to the user.
8. Eventlisteners can be added without issue. The headless logic should rely on matching the semantic html and attributes to e.g. figure out which is the next cell to focus.
9. All events are allowed to bubble up, no shadow dom, no cry.
10. Can be covered by custom renderers

[JoCa96]

POC can be found on branch joca96/poc-implement-onyx-data-grid in Storybook at http://localhost:6006/iframe.html?id=datagrid--default&viewMode=story