Expose formatting to components

[hughess]

Description

This PR adds formatting control to components. It adds a new function to take a string and return an Evidence formatting object, as well as props for each component to accept formats from users.

Closes #709

Setting a Format

The new format props accept a string. The format can be in any one of the categories below:

• Custom Format Tag
• Evidence Built-in Format Tag
• Excel-style format code

New Props

Charts:

• xFmt
• yFmt
• sizeFmt (for BubbleChart)
• valueFmt (for Sankey and Funnel Charts)

Tables

In the Column component:

• fmt

Value Component

• fmt

BigValue Component

• fmt
• comparisonFmt

Annotations

In ReferenceLine:

• ReferenceLine inherits the formatting of whichever column you're using
• Adding a valueFmt override was initially considered, but is not included in this PR

Limitations

• Formatting of date axes on charts cannot be overridden at this time. You can still pass in a format, but it will only be used for tooltip or annotation formatting

Checklist

• [x] For UI or styling changes, I have added a screenshot or gif showing before & after
• [x] I have added a changeset

[changeset-bot[bot]]

🦋 Changeset detected

Latest commit: d30f7a3a345fe481b651765d50e124f2aa5bbedd

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 5 packages

| Name | Type | | --------------------------------- | ----- | | @evidence-dev/component-utilities | Minor | | @evidence-dev/core-components | Minor | | @evidence-dev/evidence | Major | | @evidence-dev/components | Patch | | evidence-test-environment | Patch |

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
evidence-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 23, 2023 5:41pm

[netlify[bot]]

✅ Deploy Preview for evidence-development-workspace ready!

Name	Link
🔨 Latest commit	d30f7a3a345fe481b651765d50e124f2aa5bbedd
🔍 Latest deploy log	https://app.netlify.com/sites/evidence-development-workspace/deploys/6495d90fa96a2c000888a653
😎 Deploy Preview	https://deploy-preview-964--evidence-development-workspace.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site settings.

[archiewood]

This is great. Only prop I thought might be missing is sizeFmt for the bubble chart tooltips?

1. Do we want to customize the fmt function that we also expose to users to use in js so that it uses the same underlying function that accepts both string and format code.

2. We should add this to the docs

[archiewood]

Re 1: https://github.com/evidence-dev/evidence/blob/main/packages/preprocess/src/process-queries.cjs#L29

[hughess]

Ah yes, missed that one - thanks!

I didn't realize the format function was already in preprocess. I was initially going to do another PR after this one to expose that function, but that's perfect - I'll swap that out for the new function.

Will update the Number Formatting page in the docs as well.

[hughess]

There's one other remaining issue here, which is date formatting - there's a few pieces to this:

• ECharts date axis formatting cannot be overridden yet - I think this one is fine for now
• The underlying formatting functions need to know if a value is a number or a date - I've set it to number as a starting place, but that causes problems for dates
  • Possible solutions here:
    • Check the data type of the column being passed into the function
    • Modify the functions to apply the formatting code regardless of the data type
• If we have the data type check for dates, another problem is created - the timezone shift we've run into in the past (e.g., you might have May 1 on your chart axis, but the tooltip shows Apr 30)
  • This one needs some investigation

I'm going to take a look into these, but depending on their complexity, we might be better off releasing component formatting only for numbers as a starting place.

There is actually a workaround for custom date formatting as well - using custom format tags works, it's just the Excel codes directly in the function that don't work.

[archiewood]

Reading it again, I think we should try to simplify the number formatting docs.

I feel like the first line should say something like:

The easiest way to adjust formats is using component props. You can pass in excel style format codes, or one of a list of predefined format tags.

(Then an example using an excel format tag in a Value component)

And then sections with detail on:

• Formatting via column names (if you always want a column to appear that way)
• Format function (if it's outside a component)

[hughess]

Got it working!

[archiewood]

Oh thats so sick with HMR

[hughess]

Update on the date issue:

• I've modified the getFormatObjectFromString function to accept an optional valueType parameter. Where possible, I've passed this in from the columnSummary in components to guarantee that we're using the correct value type
• I've added type inference to the fmt function since it won't have access to the columnSummary

The last unresolved piece which is likely a separate issue is this timezone shift I noticed - I believe it was when manually creating dates in a duckdb query, but will confirm.

[hughess]

I've just added changes to the docs in this branch.

With this new way of using formatting, the terminology becomes a bit confusing - we were using "format tags" to describe everything, but now I think we need different terms. Here's what is in this version:

• Format tags appended in SQL --> SQL Format Tags
• Individual format tags (e.g., usd, mydate) --> Format / Format Name
• Format codes provided by Evidence (usd, eur, etc.) --> Built-in Formats
• Custom format codes (mydate, mynum, etc.) --> Custom Formats
• Manually specified format codes ($#,##0, etc.) --> Excel-style Formats
• Formatting directly in markdown ({fmt(400, 'usd')}) --> Format Function

With these changes, tag only applies to SQL Format Tags and everything else is referenced as a format (which can be built-in, custom, or Excel-style).

To Summarize

There are 3 ways to use formats in Evidence post this PR:

1. Within components (using props)
2. Within SQL (using SQL Format Tags)
3. Directly in markdown (using the format function)

And Evidence supports these types of formats:

1. Excel-style formats
2. Built-in formats
3. Custom formats

[archiewood]

In my mind, the "tag" is specifically the bit you append to the SQL column name.

So my take would be slightly different, but similar:

Example	Name	Concept
xFmt	format prop	method
fmt(400, 'usd')	format function	method
sales_usd0	appended format tag	method
$#,##0	format code	format specifier
mydate	(custom) format tag	format specifier
usd0	(built in) format tag	format specifier

Formatting methods

• Format props: you pass a format tag or format code to specify the format
• Format functions: you pass a format tag or format code to specify the format
• Column tags: you can append a format tag to a SQL column name to format it everywhere

Specifiers

• format code use excel syntax
• There are a number of built in format tags for convenience that refer to specific format codes
• You can create custom format tags

WDYT? I think it means we have less concepts?

[archiewood]

Also as side note, I think custom format tags are now much less important, as you can pass in an excel code directly to components: I might demote them in importance as a result.

Edit: so the P1 concepts we want people to understand in 30s:

• control formats in component props
• pass in excel style format codes
• there are some useful built in format tags

everything else is advanced use

[hughess]

Agreed on the custom format importance. Those P1 items make sense to me.

After opening up component props, the word "tag" doesn't seem to fit as well (I used to think of the primary usage as "tagging" a column name with a format, so for me the tag included the underscore _usd). I think I'd find it hard to distinguish between "tag" and "code" as a new user (or at least it would require a bit of explanation).

I like your usage of format code to cover the excel case.

In your layout of the terms, I would swap out the word "tag" for "name" everywhere except "column tags" - I think that would be easier for me to understand as a new user.

[archiewood]

Your tag point is a good one.

I don't love "format name" though. I think name implies human readable, whereas this usd0 is more of a shorthand for the format code.

Some other possible words

• format label
• format alias
• format keyword

None of them are perfect

[hughess]

That’s true. Format id is another option to throw into that list

[hughess]

@ItsMeBrianD I've included JSDoc on the 2 new functions that got added in this PR - let me know what you think

[hughess]

@ItsMeBrianD anything else we need to do before merging this one?

[ItsMeBrianD]

This seems like a good place to start with JSDoc, this looks good to me