Improved DataTable

[hughess]

Description

This PR is the first step in implementing improved tables in Evidence (see #468). It covers the following improvements to our DataTable component:

• Horizontal scroll: if you have more columns than the page width can display, the table with automatically allow for horizontal scroll (rather than truncate the column contents as it currently does)

• Column selection: select specific columns to include in your table by using the new Column component like so:

  <DataTable data={my_query}>
    <Column id=date/>
    <Column id=country/>
    <Column id=sales/>
  </DataTable>

  • If you don't select any columns, the table will include all columns in the query result:

    <DataTable data={my_query}/>

• Column-specific settings: using props within the Column component, you can make decisions on a column-by-column basis for the following options (more to follow in the future):
  • title: override the title of the column
  • align: left / center / right align the contents and header of the column
• New pagination approach: the data scrubber in DataTable has been replaced by page numbers and buttons shown at the bottom of the table
• Sortable Columns: you can sort columns by clicking on the column header (using sortable=true; it is on by default)
• Search: using search=true, you can add a search bar to the top of your table. This is off by default to avoid cluttering your page
• Updated data download: when you click the download button at the bottom right of your table, it will export only the columns you've supplied to the table through the Column components - we're hoping this will be helpful in cases where you have many columns in your query and only want to export what's visible to your readers
• Updated table styling options:
  • rowNumbers: true | false - turns on or off row index numbers (off by default)
  • rowLines: true | false - turns on or off borders at the bottom of each row (on by default)
  • rowShading: true | false - shades every second row in light grey (off by default)

Examples

Sample Table

Interactive Demonstration

Development Demonstration

Checklist

• [x] For UI or styling changes, I have added a screenshot or gif showing before & after
• [x] I have added a changeset

Issues with this PR

• Table generation code: there is likely a way to implement the table generation logic more concisely.
  • The problem with this implementation is it requires a lot of duplication to cover these two scenarios:
    1. Columns are passed explicitly through Column components
    2. No columns are passed (fallback to full data object)
  • I tried preparing all data in the script tag before generating the table (removing the duplicated logic in the main Svelte code), but this caused the table to flash in an unfinished state when each page was loaded
• Error handling: errors currently do not get handled in an ErrorChart as in the previous DataTable. This is because the error needs to be passed from the Column component to the DataTable, and this will take a bit more effort to get working
• Sort icon styling:
  • Ideally, the column sort icons would be vertically centered
  • We may need an icon to indicate that the columns are sortable (e.g., show both up and down errors when not in a sorted state). I left these out to keep the table headers looking clean, but it might be required for usability
  • Columns shift quite a bit when the sort button is clicked, since column widths are determined by the content visible on the page
• Pagination styling: alignment of elements needs a bit of work here. This is not an issue now, but will be in the future when we want to offer more font size/density options within tables
• Data exporting
  • Date format: dates are exported in a format that is not usable in other tools. It is currently using javascript Date object formatting, but an ISO string would be better. This can either be fixed in the table or (preferably) in the date parsing logic implemented throughout Evidence
  • Overridden column names: overridden column names are not included in data export (only raw sql column names)
• Search behaviour:
  • Works across all columns: search works across all columns of the underlying dataset. This may produce unexpected behaviour, as you can search within columns that are not visible in the table
  • Jumping content: When you navigate to a page of data with fewer records, content from lower down on the page jumps up to fill the gap left by the missing records
• Mobile search experience: the automatic zoom-in when typing in the search bar on mobile makes the experience worse. Fix for this is increasing the font size until zoom is not required, but the required font size makes the search bar stand out from the contents of the page too much
• Print formatting: print formatting can be a lot better (e.g., pagination buttons remain in print version)
  • A stopgap solution here could be to display a nicer looking “X of Y records” message rather than the pagination buttons
• Search icon styling: the search icon in development mode appears in a light grey (as expected), but it appears black in production (see vercel dev-workspace preview for an example)
• Horizontal scroll feedback: at times, it's not clear that the table has more columns than the page width can display. In the future, we may want to add some visual feedback to show that there are more columns hidden to the right (e.g., a faded white gradient on the right edge of the table)
• Content shift on last page: The contents of the page jump, but also the pagination buttons shift out from under your cursor, which is a bad experience
• Unsort column: We should have a way to unsort a column after it's been sorted. Potentially adding a third click option: Descending / Ascending / Unsorted
• Fast pagination using buttons: allow for continuous advancement of pages by holding down the page up or down buttons. This would let you scan through records quickly
• Row count: we may want to include a total row count in addition to the page count in the pagination section. Potentially at the bottom right of the table, but this would conflict with the position of the Download button
• Limits on using slots: this implementation limits our ability to use slots as a way to enable arbitrary html insertion into tables. This is because the Column component is not used as a traditional slot, but is used to update the store used by DataTable

[changeset-bot[bot]]

🦋 Changeset detected

Latest commit: d1176cff8b86ad7ff723b89cb75d14970141edd8

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages

| Name | Type | | ------------------------- | ----- | | @evidence-dev/preprocess | Minor | | @evidence-dev/components | Minor | | @evidence-dev/evidence | Major | | evidence-test-environment | Patch |

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated
evidence-development-workspace	✅ Ready (Inspect)	Visit Preview	💬 Add your feedback	Jan 6, 2023 at 9:39PM (UTC)
evidence-docs	✅ Ready (Inspect)	Visit Preview	💬 Add your feedback	Jan 6, 2023 at 9:39PM (UTC)

[archiewood]

This is a huge improvement on what we have.

Some thoughts for your consideration:

• I might want the ability to have columns fit the larger rows and row headers. See below a case where headers get truncated
• The current implementation doesn't give you the ability to access the <slot> of the Column. This makes it harder to achieve certain extensions.
• Pagination feels a bit clunky. Particularly if I want to advance lots of rows at a time. Possible things that might help:
  • A more scroll based solution, like we now have in QueryViewer
  • Ability to type in a specific page number to jump to it - ie making currentPage editable and clickable.

[hughess]

Thanks @archiewood

I'm going to take a look at the mobile search bar again, and I like your approach to abstracting the sort icons into a component - I'll add that into the new table component.

I'm not able to reproduce the header truncation issue - it should automatically add horizontal scroll even for headers only. Which browser/page width are you using?

I agree on pagination, but I think it's probably a longer-term fix because of scrolling considerations for long tables within an article and vertical scroll on mobile. I think in the future we might want to allow multiple options - e.g., pagination, vertical scroll, scrubber, etc.

[archiewood]

Hmmm I can't repro the header issue either... Maybe I had changed something else in my branch

Re pagination I was just hoping I might be able to click on where it says page 1 and type in 500, which would then jump me to page 500 or similar, vs clicking 500 times

Also, now I notice it, we might want to style 10000 as 10,000 in the pages

[hughess]

Ahh, good catch - I hadn't considered such high page numbers.

I've fixed the number formatting, but I'll need to think about the page editing idea.

If the pagination section didn't have the editable page number, would your preference be to use pagination or the data scrubber like we have in the query viewer? And does your answer change for either of these situations:

• Taking a look at your own data in a table while you're working with it / building a report
• Presenting a table to your audience for their consumption

[archiewood]

Probably depends on the size of the dataset more than these different situations. For small datasets either seems okay. For larger datasets:

• the downside of the pagination is that there's nowhere to do it quickly.
• the downside of the scrubber is that is can jump too much with little horizontal movement

I think the scrubber is probably better overall. Though I think would probably prefer the scrubber to sit vertically up the right hand size rather than the bottom

[mcrascal]

@hughess @archiewood this is really sick.

Pagination styling: alignment of elements needs a bit of work here.

Print formatting: print formatting can be a lot better (e.g., pagination buttons remain in print version) A stopgap solution here could be to display a nicer looking “X of Y records” message rather than the pagination buttons

I took a stab at the alignment and printing issues identified in the original PR, and adjusted the colouring. I'll open a PR into this branch w/ the proposed changes for your consideration. I've included some screenshots and discussion below.

In addition to the already identified issues in the PR, I would add two more. Neither of these are blocking in my opinion.

• The content shift when you reach the last page of a table. I find this even more jarring than the content jumping when you are searching because the pagination buttons move out from under your cursor.
• Adding a way to "unsort" a column would be nice, for example the third click on the column could remove the sort. Descending / Ascending / Unsorted.

Pagination Styling

• Left aligns the pagination block
• Takes the blue out of the active pagination buttons.
  • I like the strong indication of "available-to-click", but I found them a lot more eye catching than the table content itself.
  • Not married to this if you think the grey is too soft.
• Transitions the buttons back to blue when the table is hovered - this works with "thumb on the table" when scrolling ios as well.
• Fades in the download button on hover, syncs the timing to the colour transition above
• Vertically centres pagination text, icons, download text, and download icon
• Matches text sizes between the two
• Adds "x of y records" when printing, in place of the buttons
• Incorporates the changes to the download button into how it's presented under charts (fades in, vertically centred)

Table hovered state

Before

After

Un-hovered state

Printing

[hughess]

@mcrascal looks really nice - I like it!

[hughess]

Just played around with the hovering - looks much cleaner. Liking the fade in on the download button too

[hughess]

I played around with the idea Archie had to allow an input for the page number (and using the hover feature Adam added).

@archiewood @mcrascal what do you guys think of this?

The input width is tricky to set - would need to try it with larger page numbers and see what our options are.

[mcrascal]

I dig it. Seems discoverable without being intrusive when you're just reading.

One other idea for a future release: I think it would be nice to advance the pages continuously when you hold down the next/previous page button. I often use the scrubber to visually scan data that way, and I find it demos really well for teams who are moving off of a slower tool.

[archiewood]

Yeah I really like that.

Could have a tiny bit more spacing after the "/" before the "total pages" number to even it out? Not a biggie though.

[hughess]

Ok I made some modifications and got it working more consistently. I've set it up to show 3 digits, which I hope will cover most scenarios.

This isn't great for very large page numbers, but I think we'll need a better solution for those anyways, so this can be a good option until we have that.