We're currently working on native support for tokens internally here at Figma. In our eyes there are two core use cases that stem from customer requests for design tokens:

Token aliasing (i.e. danger-bg -> red-300)
Theming

Currently the spec does not support theming, which at the moment is a blocker for us for full adoption. I'd like to start a thread here on what native mode support would look like for this format. Major shout out to @drwpow for trailblazing some of this with Cobalt-UI, to @jkeiser for turning this into a proper proposal, and to @connorjsmith for comments and critiques.

Here's the proposal we ended up with:

Overview

Modes represent alternative sets of values for a collection of design tokens. For example, one might wish to have a different value for the “background” and “foreground” tokens depending on whether they are in “light” mode or “dark” mode.

This proposal allows the user to define a set of modes that apply to all tokens in the design tokens file, allowing them to have distinct values for each mode.

Herein we’ll use this example:

{
  "$name": "Figma UI Colors",
  "$modes": {
    "light":      {}, // no fallback
    "dark":       {}, // no fallback
    "super-dark": { "$fallback": "dark" }
  },
  "bg": {
    "$type": "color",
    "brand": {
      "$value": "{colors.blue.300}", // light mode falls back to this
      "$modes": {
        "dark": "{colors.blue.500}" // super-dark mode falls back to this
      }
    }
  },
  "fg": {
    "$type": "color",
    "brand": {
      "$modes": {
        "light": "{colors.black}",
        "dark": "{colors.white}",
        "super-dark": "{colors.gray}"
      }
    }
  }
}

In this example, the values for bg and fg for each mode would be:

	light	dark	super-dark
bg	{colors.blue.300}	{colors.blue.500}	{colors.blue.700}
fg	{colors.black}	{colors.white}	{colors.white}

Defining Modes

A design tokens file may optionally define a set of named modes at the top of the file.

{
  "$name": "Figma UI Colors",
  "$modes": {
    "light": {},
    "dark": {},
    "super-dark": { "$fallback": "dark" }
  },
  // tokens ...
}

The $modes definition is an object at the top level of the design tokens file.

$modes should be placed before the first token or token group, to make efficient file import possible.
Mode names are case sensitive: "``light``" and "``LIGHT``" are different modes.
Mode names have the same restrictions as token and group names: they must not start with $, and must not contain {, . or } characters.
If $modes is empty {}, it is treated the same as if it were not specified (namely, that no modes are defined).

Fallbacks

Each mode may optionally define a $fallback mode, which will be used to determine the value of tokens which do not define a value for the given mode.

The lack of a $fallback value implies that mode will fall back to a token’s default $value.
```
    "dark":       {}, // no fallback
```
$fallback value must be the name of another mode in the same file.
```
    "super-dark": { "$fallback": "dark" }
```

Fallbacks must not form a cycle.

    "dark": { "$fallback": "super-dark" },
    "super-dark": { "$fallback": "dark" } // ERROR: cycle

Token Values

Design token files may specify different values for each mode, for each token.

    "brand": {
      "$value": "{colors.blue.300}", // light mode falls back to this
      "$modes": {
        "dark": "{colors.blue.500}" // super-dark mode falls back to this
      }
    }

A token may optionally define $value, which determines its default value.
If no modes are defined, $value must be defined and represents the token’s value.
A token may optionally define $modes, which is an object defining its value for specific modes.
A token’s $modes may only define values for modes defined in the same file. "$modes": {"daaaaark":"#000000"} is an error if there is no "daaaaark" mode.
"$modes": {} is equivalent to not specifying $modes at all.

NOTE: this relaxes the requirement that $value is required when modes exist.

Value Resolution

If modes are defined, all tokens must have values for all modes, taking into account fallback and default rules. This means that either $value or $modes (or both) must be defined for all tokens.

The value of a token for mode "m" is as follows:

If the token defines a value for "m" , that value is used for the mode.
Otherwise, if the mode defines a $fallback, the token’s value for the fallback mode is used. The same rules are applied for the fallback mode, so if an explicit value is not defined for the fallback mode, its fallback is used, and so on.
Otherwise, if $value is defined, then that value is used for the mode.
Otherwise, the token is undefined for the mode, which is an error.

I like this approach. I am curious if mode nesting in token definitions would be supported or explicitly disabled?

For example, you have light and dark mode, but for each of those you also have increased and decreased contrast modes. Modes in this case could not be defined in isolation as there is some dependency (light + increased contrast, dark + increased contrast).

Example contrast modes:

{
  "$name": "Figma UI Colors",
  "$modes": {
    "light": {},
    "dark": {},
    "increased-contrast": {}
    "decreased-contrast": {}
  },
  // tokens ...
}

Examples with nesting:

  "text-primary": {
    "$type": "color",
    "brand": {
      "$modes": {
        "light": {
          "value":  "{colors.gray.800}",
          "increase-contrast": "{colors.gray.900}",
"decreased-contrast": "{colors.gray.700}"
        },
        "dark": {
          "value":  "{colors.gray.200}",
          "increase-contrast": "{colors.gray.100}",
"decreased-contrast": "{colors.gray.300}"
        }
      }
    }
  }

Alternatively, disallowing nesting for modes may provide a forcing function for aliasing (one level handles light/dark, another level handles contrast). Either I think are ok but should be considered.

Similarly, it may be necessary to define which modes are relative to one another, or what token types they can support. Ie, "light", "dark", and "darkest" are enumerations for a property (eg, "colorScheme"), whereas "increase-contrast", "decreased-contrast" or even "forced-colors" would be enumerations for a different property (eg. "contrastModes"). That way we can enforce that you cannot nest/combine options of the same mode property (eg, "light": {"dark": "$token"}} is disallowed).

Today for theming we just add a new theme file with the tokens defined in the theme file overriding said tokens in the base file. This allows product teams to create custom themes without messing with the core token file.

With the modes proposal, I do not see a mechanism for defining a mode outside the main token definitions, or would it work the same way? Define the same tokens a second time with just the additional mode value?

On Thu, 23 Mar, 2023, 9:02 am Nate Baldwin, @.***> wrote:

I like this approach. I am curious if mode nesting in token definitions would be supported or explicitly disabled?

For example, you have light and dark mode, but for each of those you also have increased and decreased contrast modes. Modes in this case could not be defined in isolation as there is some dependency (light + increased contrast, dark + increased contrast).

Example contrast modes:

{ "$name": "Figma UI Colors", "$modes": { "light": {}, "dark": {}, "increased-contrast": {} "decreased-contrast": {} }, // tokens ... }

Examples with nesting:

"text-primary": { "$type": "color", "brand": { "$modes": { "light": { "value": "{colors.gray.800}", "increase-contrast": "{colors.gray.900}", "decreased-contrast": "{colors.gray.700}" }, "dark": { "value": "{colors.gray.200}", "increase-contrast": "{colors.gray.100}", "decreased-contrast": "{colors.gray.300}" } } } }

Alternatively, disallowing nesting for modes may provide a forcing function for aliasing (one level handles light/dark, another level handles contrast). Either I think are ok but should be considered.

Similarly, it may be necessary to define which modes are relative to one another, or what token types they can support. Ie, "light", "dark", and "darkest" are enumerations for a property (eg, "colorScheme"), whereas "increase-contrast", "decreased-contrast" or even "forced-colors" would be enumerations for a different property (eg. "contrastModes"). That way we can enforce that you cannot nest/combine options of the same mode property (eg, "light": {"dark": "$token"}} is disallowed).

— Reply to this email directly, view it on GitHub https://github.com/design-tokens/community-group/issues/210#issuecomment-1480552982, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEKS36B4K5ZHJNAK7WWVIKDW5O76PANCNFSM6AAAAAAWEORCFY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

This proposal is missing a critical bit. Since modes are user provided content translation tools can not generate code based on modes.

All possible modes must be defined by the specification for translation tools to work.

see : https://github.com/design-tokens/community-group/issues/169

Today for theming we just add a new theme file with the tokens defined in the theme file overriding said tokens in the base file. This allows product teams to create custom themes without messing with the core token file. With the modes proposal, I do not see a mechanism for defining a mode outside the main token definitions, or would it work the same way? Define the same tokens a second time with just the additional mode value?

Depends on the heuristic we'd want to use for what constitutes an "extension" of the base theme (either via some sort of $extends, or just via name matching), but I'd think that defining more modes would be something like this

File 1

{
  "$name": "Figma UI Colors",
  "$modes": {
    "light": {},
    "dark": {},
    "increased-contrast": {}
    "decreased-contrast": {}
  },
  // tokens ...
}

File 2

{
  "$name": "Figma UI Colors Extended",
  "$extends": "Figma UI Colors", // Alternatively tools just batch accept files and try to combine them blindly
  "$modes": {
    "extra-dark-mode": {"$fallback": "dark-mode"}
  },
  // tokens (can overwrite values for light and dark mode, as well as define new values for the added extra-dark-mode)
}

This proposal is missing a critical bit. Since modes are user provided content translation tools can not generate code based on modes.

All possible modes must be defined by the specification for translation tools to work.

see : #169

Could you expand on this? This proposal defines all modes upfront inside $modes, or am I misunderstanding the problem?

A minimal example would be super helpful, thanks in advance!

A minimal example would be super helpful, thanks in advance!

The json in the first post is an excellent example.

How would a translation tool process that file? How do end users "access" the mode specific values?

Taking CSS as an example. What would the generated CSS code be for that file?

I would expect the following generated CSS

/* figma-ui-colors_light.css */
:root {
  --bg-brand: #1010FF; /* colors.blue.300 */
  --fg-brand: #000000; /* colors.black */
}

/* figma-ui-colors_dark.css */
:root {
  --bg-brand: #1010FF; /* colors.blue.500 */
  --fg-brand: #FFFFFF; /* colors.white */
}

/* figma-ui-colors_super-dark.css */
:root {
  --bg-brand: #0000FF; /* colors.blue.700 */
  --fg-brand: #A0A0A0; /* colors.gray */
}

Usage:

@import url("figma-ui-colors_light.css") prefers-color-scheme: light
@import url("figma-ui-colors_dark.css") prefers-color-scheme: dark

Those variables would be split up over 3 files, one for each mode. Alternatively a generator could specify all modes in a single file

:root {
  --bg-brand: #1010FF; /* colors.blue.300, from $value */
}

:root[mode="light"] {
  --bg-brand: #1010FF; /* colors.blue.300 */
  --fg-brand: #000000; /* colors.black */
}

:root[mode="dark"] {
  --bg-brand: #1010FF; /* colors.blue.500 */
  --fg-brand: #FFFFFF; /* colors.white */
}

:root[mode="super-dark"] {
  --bg-brand: #0000FF; /* colors.blue.700 */
  --fg-brand: #A0A0A0; /* colors.gray */
}

Of course tools could probably optimize the above to utilize css var override cascades

Another alternative would be for tools to append the mode to token names, should a "multi mode" or override use case be required

  --bg-brand-light: #1010FF; /* colors.blue.300 */
  --fg-brand-light: #000000; /* colors.black */

  --bg-brand-dark: #1010FF; /* colors.blue.500 */
  --fg-brand-dark: #FFFFFF; /* colors.white */

  --bg-brand-super-dark: #0000FF; /* colors.blue.700 */
  --fg-brand-super-dark: #A0A0A0; /* colors.gray */

Are there other gaps that a generator tool wouldn't be able to create based on the proposed JSON structure?

Looking at the postcss plugin, I'd expect this to be possible with an added mode() extension. The mapping could be done in postcss config as well, similar to the is option

Using new mode() option

@design-tokens url('./figma-ui-colors.json') format('style-dictionary3') mode('light');
@design-tokens url('./figma-ui-colors.json') when('brand-2') format('style-dictionary3') mode('dark');

.foo {
  color: design-token('fg.brand');
}

Using postcss config

postcssDesignTokens({ 
  modes: { 
    'figma-ui-colors.json': 'dark'
  } 
})

I think there is some confusion here :) Is this a proposal for "theming" or for "conditional/contextual values"?

Theming :

one design system
multiple sets of values
multiple outputs
static result

Examples :

a customizable design system that is used by 3rd parties
having variants of a design in multiple color hues (a blue theme, a red theme, ...)
...

Conditional values :

one design system
multiple sets of values
one output
dynamic result
native API's

Examples :

dark mode
light mode
high contrast mode
large screens
small screens
...

It might sometimes be possible to build a dynamic result by combining multiple themes on a single page or screen but these are always custom mechanics that don't leverage native API's.

Given the mentions of dark mode and high contrast mode I assumed this was a proposal for conditional values.

Can you clarify?

If this is a proposal for theming and isn't intended for dark mode / light mode then we would just wire this up behind is in the PostCSS plugin.

For a tokens file with modes a and b defined within a single file :

@design-tokens url('./figma-ui-colors.json') format('style-dictionary3');

.foo {
  color: design-token('fg.brand');
}

output A :

postcssDesignTokens({ 
  is: ['a']
})

output B :

postcssDesignTokens({ 
  is: ['b']
})

But that is only interesting for us to do when this feature isn't intended for dark / light mode and other conditional values.

This proposal doesn't make it possible for us to support this :

CSS author writes :

.foo {
  color: token('foo');
}

We generate :

.foo {
  color: red;
}

@media (prefers-color-scheme: dark) {
  .foo {
    color: pink;
  }
}

We can not generate this output because there isn't any link between a mode with user defined name "dark" and the concept of a functional dark mode.

That kind of link can, however, be supplied to the code generator as configuration. Every code generator tool is going to require some amount of configuration to make the output fit well with your codebase; a per-output-platform mapping of mode names to "the native thing" could be part of that configuration.

{
  "light": "",
  "dark": "@media (prefers-color-scheme: dark)"
}

Put another way, even if this spec defined a full set of what the allowed themes/modes were, and they were light and dark, a CSS code generator would still need to know how to map those themes to code. One project wants @media (prefers-color-scheme: dark), another makes dark mode the default and uses @media (prefers-color-scheme: light), and another uses [theme-dark] and switches attributes with JavaScript.

[theme-dark] That is not a native API for dark mode and shouldn't be taken into consideration. Anyone can do any custom thing.

One project wants @media (prefers-color-scheme: dark), another makes dark mode the default and uses @media (prefers-color-scheme: light)

That would just work. Your base set of values wouldn't be wrapped in any conditional at rules. Your conditional set of values would be wrapped in whatever conditional matches.

If a designers decides to do dark as default then the code generator would produce :

.foo {
  color: pink;
}

@media (prefers-color-scheme: light) {
  .foo {
    color: red;
  }
}

But I don't want to focus to much on conditional values and the benefits of being to generate those without first confirming if the original proposal was for theming or conditional values.

Sorry for the confusion! Yes this is purely about theming/modes, not with determining how/when those modes should be applied.

Since modes are user provided content translation tools can not generate code based on modes.

All possible modes must be defined by the specification for translation tools to work.

I heavily disagree with this. Any list that we come up with of valid modes will never satisfy the demands of end users. Let's say for example we have these modes as valid ones:

Light
Dark

But now a user wants to add midnight mode (i.e. similar to dark, but all near-blacks are set to black to preserve battery life on OLED mobile devices, a fairly common pattern). Are we saying that midnight modes are an invalid use case for tokens?

Maybe we add it, so now our list is:

Light
Dark
Midnight

But now a user wants to add high contrast. Another user wants to add colorblind mode. Lets say somehow we hypothetically come up with a list of modes that encapsulate all possible visual modes (which personally I don't believe is possible, but lets say hypothetically we did) and we ended up with something like,

Light
Dark
Midnight
Deuteranopia
Protanopia
Tritanopia
High Contrast
Low Contrast
...etc

But a large organization wants different themes per product brand, one of which has a light appearance with a red brand color, and another of which has a dark appearance with a green brand color. Are we saying that's an invalid use of tokens?

Modes need to be user defined, not an enum.

The output of these modes by translation tools should attempt their best-guess, but there are hundreds of ways people represent theming on the web today - there's no standard for it (aside from prefers-color-scheme, which we already know doesn't satisfy the needs of users).

That kind of link can, however, be supplied to the code generator as configuration. Every code generator tool is going to require some amount of configuration to make the output fit well with your codebase; a per-output-platform mapping of mode names to "the native thing" could be part of that configuration.
{
  "light": "",
  "dark": "@media (prefers-color-scheme: dark)"
}

I like this, though this also feels like something that might belong more in $extensions rather than the core format, though I'm curious what others think here. IMO the core format should be apathetic to the output format.

@jjcm https://github.com/design-tokens/community-group/issues/210#issuecomment-1481853876

Please see my comment here : https://github.com/design-tokens/community-group/issues/210#issuecomment-1481554248

Why not have both generic themes and conditional/contextual values?

The core concept of this design tokens specification is to be a bridge between different formats. Part of that work imo is mapping concepts like native API's for dark mode.

IMO the core format should be apathetic to the output format.

I disagree with this.

The format should try to be platform agnostic but it must be sympathetic to how it will be used.

If a small change to the format can make a large difference on the output for all platforms then that seems like a worthwhile thing to me.

I really like this proposal and think building theming into the format will improve interoperability.

At Interplay, we use a very similar data structure internaly and get users to import/export specific themes to the token group format. The only difference is we use $values rather than $modes, and $inherit rather than $fallback

One suggestions I have is making $fallback an array instead of a single value.

Then users could define themes/modes for compact, jumbo, dark and light. Then they can combine these to create e.g. dark-mobile, light-mobile, etc. The value of the tokens would be determined by the order of the inherited themes.

This would save a lot of duplicate entry for values across themes.

{
  "$name": "UI Colors",
  "$modes": {
    "light": { }, 
    "dark": { },
    "compact": { },
    "jumbo": { }, 
    "dark-compact": { "$fallback": ["dark", "compact"] }
  },
  "font-size": {
    "$type": "dimension",
    "small": {
      "$value": "1rem",
      "$modes": {
        "compact": "0.5rem",
        "jumbo": "1.5rem",
      }
    }
  },
  "backgrounds": {
    "$type": "color",
    "base": {
      "$value": "{colors.white}",
      "$modes": {
        "light": "{colors.white}",
        "dark": "{colors.black}",
      }
    }
  }
}

The values for these tokens in the dark-compact theme would be font-size.small: 0.5rem backgrounds.base: {colors.black}

@mkeftz interesting proposal with the array, totally see the use case.

One thing I'm curious about though is if others expect to define their dimension modes along with their color modes. Having compact be a valid theme for UI Colors feels somewhat weird to me. I'd expect each of these to be in separate files given the lack of overlap in most situations. Would be curious to see what others think.

Perhaps mobile/desktop and compact/large would be a more common combo? E.g. certain buttons would likely want their mobile sizings (for us fat fingered individuals), but non-interactive densities would want to stay compact (with the occasional explicit mobile-compact value where those two would overlap).

Mainly spitballing though, I'd also be curious to see how other people would plan on using that!

This topic requires a lot of research, or basically the one I did for the past years and started to put into blog posts. I'm currently nearing ~10k words (so there is a substantial backup for what's to come). I hope to start releasing this series in late april.\ With the spoiler out of the way that there is more to come, I think this topic is very important and I can share parts I have ready already.

Features / User Preferences

Let's start this from user preferences, this is what a user might wanna choose:

user-preferences

And the idea is, whatever the user is about to choose, will receive the respective value for that token. As theme authors we would call them features. A theme supports feature color-contrast or color-scheme, etc. Not all of these can be "System", as these may be product related options. See next section for more differentiation.

By the way, here is github:

They don't have a "skin", but support all the other features from the user preferences menu above (in terms of color). You can even have dark appearance set when your system is light o_O.

Behavior

There is a behavior involved in here:

Adaptive: User chooses "System" and passes the choosing on to the Operating System
Mode: User explicitely chooses a particular preference

In CSS:

:root {
  --background: white;
}

/* Adaptaive */
@media (prefers-colors-scheme: dark) {
  :root {
    --background: black;
  }
}

/* Mode */
[theme-color-scheme="dark"] {
  :root {
    --background: black;
  }
}

The important thing: The behavior goes independent from the storage of values in the tokens file! This goes into the next step when values are extract from the tokens file and put into CSS or whatever else format

References as Solutions in Development

In development we use references as solutions to this problem, here is one for colors:

With that configuration above that is:

intent-action-base-background = 4 x 4 x 3 x 3 = 144

so the token can take 144 permutations in this example - if there was a value for all of them (which in reality I wouldn't expect).

A Format to Structure Permutations

Modes is the wrong word to this - as this is something a user would opt into (see Raskin, J. (2008). The Humane Interface: New Directions for Designing Interactive Systems (10th printing). Crawfordsville, Indiana: Addison-Wesley.) For example nowadays it would be dark color-scheme and not dark mode (that goes back to the old days, when it really was a mode - it still is but has moved to OS level).

I'm also not sure yet, what would be the best format to support this. It however needs to be defined on the token itself. It needs to be stored alongside the feature configuration. For example:

{
  "$name": "intent-action-base-background",
  "$value": [
    {
      "$value": "white"
    },
    {
      "$value": "darkgrey",
      "color-scheme": "dark"
    },
    {
      "$value": "black",
      "color-scheme": "dark",
      "color-contrast": "high"
    }
  ]
}

The finding of tokens would be programmatical. Those with the highest match win:

const token = tokens.find(t => t.name === 'intent-action-base-background');
const value = token.value.find({ 
  colorScheme: 'dark', 
  colorContrast: 'high' 
});

That is in fact no different than how tools currently work, as danny wrote in https://dbanks.design/blog/dark-mode-with-style-dictionary (Single Token Method)

An alternative to stick with the object approach would be (DON'T DO THIS):

{
  "$name": "intent-action-base-background",
  "$value": {
    "": "white",
    "dark-": "darkgrey",
    "dark-high": "black"
  }
}

where features become the key within $value (This is how one needs to do that in figma right now, because styles don't support multiple values... hint hint :D). However, I'm heavily against this, as it would eliminate structural information and requires parsing.

In terms of scalability a format is required that can potentially scale up to ~150 permutations per token but we also know there will be maybe permutations based on 2 features with 2 permutations - as companies will already started to support color scheme and soon color contrast is to come (maybe because it only has two options? or because it has media query backup?). But once this wave is over, we probably will see adoption for chroma - Yes I'm very hypothetical here.

On the other hand the format shall be practical, this is why all (?) ideas so far use an object. This could easily end up in a nightmare nested tree as @NateBaldwinDesign showed (given the github use-case above) and practicability is gone.

I'm still with the array and it's configuration, but happy to read about better ideas.

Also, if I rushed over some of the topics, then please ask for more detailed explanation.

so the token can take 144 permutations in this example - if there was a value for all of them (which in reality I wouldn't expect).

Definitely agree! That's one of the reasons why we are pushing for fallback definitions for each mode, as well as overrides being optional. You shouldn't have to define all permutations.

Modes is the wrong word to this

Modes is the wrong word if you're only supporting color. What complicates things is that design tokens are used for many use cases, color just being one of them. We originally were using theme and color-scheme, but we found it didn't apply for other use cases of tokens. Here are the use cases we came up with that we felt the terminology should support:

visual themes (i.e. light/dark)
product/brand themes (i.e. docs/sheets/slides)
accessible themes (high contrast, colorblind modes)
dimensions and platform size differences (mobile/desktop/tablet)
UI string translations / alternatives (i.e. english/spanish/german)
and likely more in the future! What happens with depth is a factor for VR/AR?

In the end we found that color-scheme simply didn't work outside of color (i.e. a mobile color scheme doesn't convey that you're talking about differences in padding), anything that's specific like that also encounters issues of not being scalable for future use cases. theme was great for color as well, but it didn't work well for things such as translations (i.e. a "German theme" feels like you're going to have a bavarian/oktoberfest vibe, not having strings translated to German). We ran a survey among a few different disciplines and users, and found that mode was the least-bad option for a generic term. It was the 2nd choice among the 3 cohorts we tested: visual designers, content designers, and those working explicitly on dimensionality - not their first choice, but one that still made sense for their use case.

The permutation structure you suggest is interesting - treating each permutation as a flag rather than an individual grouping. I'm not a huge fan of the readability of it (though I'm not sure if that matters for an interop format), but I see the value. Would be curious on the needs of this vs explicit modes. One thing to call out is with this approach you'd need some very explicit logic around missed finds, as with this approach it's possible for no correct value to exist.

As an example, if you had:

{
  "$name": "intent-action-base-background",
  "$value": [
    {
      "$value": "white"
    },
    {
      "$value": "darkgrey",
      "color-scheme": "dark"
    },
    {
      "$value": "black",
      "color-scheme": "dark",
      "color-contrast": "high"
    }
  ]
}

And you queried for:

const token = tokens.find(t => t.name === 'intent-action-base-background');
const value = token.value.find({ 
  colorContrast: 'high' 
});

Would you return black or white in this case? There's no explicit definition for color-contrast: high without dark mode defined. Do we then return the dark mode value? Or do we return the light value?

If invalid queries like this aren't allowed, validations would be non-trivial as they'd be exponential in nature to test. A flag based approach is definitely going to need more thought.

Thanks for sharing this thorough proposal @jjcm, and thanks everyone else for all the insightful comments here!

Overall, I really like this proposal. I think it would provide the format with an elegant mechanism for providing alternate token values for any number of use-cases. As Jake pointed out in the previous comment, this could have applications way beyond light and dark color schemes.

Of course, since the modes are author-defined (i.e. there wouldn't be a pre-defined set of permitted mode names in the spec), tools can't automatically infer any semantics from the mode names or apply specialised functionality based on the selected mode.

If I understood correctly, I think this is essentially the issue @romainmenke raised in his comments. For example, a translation tool could not "know“ that a particular mode represents the colors that should be used when the user has set their OS to dark mode and therefore could not automatically output an appropriate @media query in CSS.

However, I think the $modes proposal nonetheless enables a lot of use-cases that are so far impossible to represent using the current DTCG format draft. I'd therefore be in favour of adding it to the spec.

I do think some of the finer details need ironing out first though...

Does `$value` really need to be optional?

The OP proposal relaxes the requirement for every token to have a $value since could instead just have $modes. While this certainly could be done, I wonder whether it might keep things a bit more consistent and simple if we keep $value being mandatory.

The way I see it, every DTCG file has an implied "default" mode, which is what you get if you didn't use $modes at all. So why not lean into that and encourage authors to only use $modes for additional modes they might need.

Taking the initial example from the OP, we might decide that "light" is our default and therefore only use $modes to create the additional "dark" and "super-dark" modes. The example could then be rewritten as:

{
  "$modes": {
    // no longer specify light
    "dark":       {}, // no fallback
    "super-dark": { "$fallback": "dark" }
  },
  "bg": {
    "$type": "color",
    "brand": {
      "$value": "{colors.blue.300}", // light mode falls back to this
      "$modes": {
        "dark": "{colors.blue.500}" // super-dark mode falls back to this
      }
    }
  },
  "fg": {
    "$type": "color",
    "brand": {
      "$value": "{colors.black}", // use $value rather than $modes.light here
      "$modes": {
        "dark": "{colors.white}",
        "super-dark": "{colors.gray}"
      }
    }
  }
}

I think doing it this way and retaining the "every token must have a $value" rule is preferable for the following reasons:

Avoids situations where an author omits $value but forgets to specify $modes values for every mode. In that case there'd be no $value to fall back on, so the token becomes... invalid?
Could make the learning curve for the format lower since there's less exceptions to learn. I think "every token must have a $value" is simpler than "every token must have a $value, unless it has a $modes property". Especially if you consider $modes an advanced feature that newcomers might not learn about right away.
Keeps parser logic a (tiny bit) simpler too for the same reason as above.

I think the only thing we might lose would be the ability to assign a name the default mode. In the above example, users would need to know that "no mode means light mode". However, we could solve that by having a means to provide optional metadata about the default mode. For example, we could have an optional $defaultMode property that could be added to the top-level $modes declaration like so:

{
  "$modes": {
    "$defaultMode": {
      "$name": "light"
    },
    "dark":       {}, // no fallback
    "super-dark": { "$fallback": "dark" }
  },
 // ...
}

Could modes have additional metadata?

For example $description? I could imagine this might be useful for tools that have the ability to list out all declared modes to the user.

{
  "$modes": {
    "$defaultMode": {
      "$name": "light",
      "$description": "A color scheme to be used when the user has expressed a preference for a light UI"
    },
    "dark": {
      "$description": "A color scheme to be used when the user has expressed a preference for a dark UI"
    },
    "super-dark": {
      "$description": "A even darker version of the dark color scheme which uses pure black backgrounds to preserve energy on devices with OLED displays",
      "$fallback": "dark" 
    }
  },
 // ...
}

It's conceivable that future spec iterations might add more properties for other bits of useful metadata for tokens and groups. When doing so, we could also consider whether they might be useful for mode declarations and, if so, permit them to be used here.

Should modes have `$extensions`?

Sort of related to the previous point, maybe permitting $extensions on mode declarations could be useful too. The use-case would be extensions that relate to a mode itself rather than a token or group.

This kind of thing could perhaps enable tools to add some additional semantics per mode, as @TravisSpomer was suggesting in response to @romainmenke.

E.g.:

{
  "$modes": {
    "$defaultMode": {
      "$name": "light",
      "$extensions": {
        "org.postcss.design-tokens.media-query": "(prefers-color-scheme: light)" 
      }
    },
    "dark": {
      "$extensions": {
        "org.postcss.design-tokens.media-query": "(prefers-color-scheme: dark)" 
      }
    },
    "super-dark": {
      "$extensions": {
        "org.postcss.design-tokens.media-query": "(prefers-color-scheme: dark) and (prefers-contrast: more)" 
      },
      "$fallback": "dark" 
    }
  },
 // ...
}

Obviously, the standard $extensions caveat of being prorietary and thus not universally supported by tools applies. But this could still provide an "escape hatch" to enable tool-specific functionality. Also, if particular extensions gain a lot of adoption, it provides us with a cowpath to pave in future spec iterations.

Oh, so wonderful thoughts in your answer :)

PS. While typing my answer @c1rrus also posted (I will read his post after I posted mine). This is a response to @jjcm

I think, permutation (as odd as it sounds) is the word I used most for to describe those varying token values. It is not as much as opinionated as mode - wdyt? Perhaps not necessary, as values might be best as an array?! 😅

I thought about validation and fallback a bit more after posting here. Here is a bit more of what's in my head.

Do we want to have a fallback for the sake of having a fallback, because we want to have it or because it must be there? I think we are going with fallback, because we want to have it to avoid dealing with the complexity that awaits us. And is fallback even a correct value then?

About features, these are defined by your product/theme/design system - they may or may not live within the token file. The bonus of having it is to that you can validate the file by itself, which I'd actually in favor of having - or this can be set as reference:

{
  "$reference": "path/to/features.tokens"
}

I think, having validation is important. I'm author of theemo and am currently working on v2 which actually is about letting designers define their features and then provide a UI for letting them define values for a token based on features - sounds familiar?

Here is the challenge: Let's say you defined two features, color-scheme (light/dark) and color-contrast (low/high) and you have a color token with a given value. Now you want this token to support color-scheme. Your UI splits into two input fields one for light one for dark and copies over the value from before into the two new input fields. Let's say the value from before was set for light color-scheme, the designer now will choose a value for dark color-scheme. The token stays valid as long as there are two values present. The original value before this token support color-scheme feature is no longer relevant (=> no fallback needed, the fallback became the light color-scheme value). The UI can validate if the token has support for all values based on the features the token shall support.

Next up: Let's make this token support color-contrast feature. We will be presented with a 2 x 2 input matrix. Same drill again for the designer. However, if low contrast was considered the default, then the designer would only change the values for high color-contrast. The values for low color-contrast can even be omitted. This is the "none" case in the picture above. The fallback here are the two color-scheme values without color-contrast.

That is a token is valid as long as permutations of a supported feature are matched with permutations of other supported features - hell, this is a heck to explain in words. I'm having truth tables in my mind here, where you can cluster groups.

What's important to note here is, that features will have a default, which would ease building UIs and help validate (from my research before, I never find a case where there is no default, this will always ever be provided by someone - the OS at last). Let's say our default for color-scheme is light and revisit the process from above. The designer would choose to support color-scheme for a token, the previous value is copied into light color-scheme (the default) and leaves the designer to fill out only the dark value. Same for color-contrast set to low as default, when supporting color-contrast on the token, the UI would only show the option to the designer to fill out values for high color-contrast.

Which brings us back once more to fallbacks: The fallbacks are the default values of a feature.

That is a fallback for the entire value set is wrong, think this: If you have a fallback value (that was set for light color-scheme and low color-contrast) and as a user you want to experience a product in dark color-scheme and high color-contrast. If there is a value given for dark color-scheme but low color-contrast and the fallback value - which one to serve to users?

Which brings us to your question:

Would you return black or white in this case? There's no explicit definition for color-contrast: high without dark mode defined. Do we then return the dark mode value? Or do we return the light value?

I think there are two options here to answer this:

That's an invalid token, as the request you are having cannot be answered - because explicitely matching your query does not work
Assume the default for colorScheme and combine it with the query for colorContrast to return the appropriate value

When authoring (build time) we are in need of defining a default (this is truly a mode ^* here). Whereas in experiencing a product (runtime) as a user, a preference is given (by the the default from the product, by the platform (browser) or latest by the OS). With that preference present the correct token value can be found.

^* maybe this was the case for calling it mode?

I have been playing around with typing this to work on theemo - as typing is a good way to prototype this and figure out problems. In the next branch (I gave it a push right now), there are types in @theemo/core - I've been playing with only the features the web offers through media queries (color-scheme and color-contrast), but potentially could be more.

A theme in theemo would be to it's current state defined as:

{
  "name": "super theemo",
  "features": {
    "color-scheme": {
      "behavior": "adaptive",
      "default": "light"
    },
    "color-contrast": {
      "behavior": "mode",
      "default": "low"
    }
  } 
}

behavior is for the implementation on the product (ignore, see post above) but default will be used for constructing the UI for designers to pick a proper value as well as the algorithm to find the correct token value.

Could modes have additional metadata?

For example $description? I could imagine this might be useful for tools that have the ability to list out all declared modes to the user.

Should modes have $extensions?

Sort of related to the previous point, maybe permitting $extensions on mode declarations could be useful too. The use-case would be extensions that relate to a mode itself rather than a token or group.

This kind of thing could perhaps enable tools to add some additional semantics per mode, as @TravisSpomer was suggesting in response to @romainmenke.

Both of these suggestions seem like sensible considerations to add, as both fit existing patterns within the spec.

The use of $extensions to handle generator considerations seems like a particularly appropriate approach, as how tokens are output may have different solutions even within a particular platform or framework. Consider how one might handle modes and themes in CSS, there are several strategies one could employ. Just off the top of my head, one could:

Generate a single CSS file containing all permutations; each permutation wrapped within an appropriate media query
Generate a single CSS file containing all permutations; each wrapped within a rule and use some matchMedia logic to toggle rules as needed
Generate CSS files for each permutation and use the media attribute of the link or style elements to responsively load files
Some combination of all of the above

Moreso, one might desire to optimize the token output any number of ways, perhaps choosing to output a set of default/base tokens that are stable across all permutations paired with streamlined sets per permutation that only contain the unique tokes for that permutation OR simply render all tokens per permutation. This choice may even be different per targeted platform, technology, or framework.

It seems prudent to me, that the DTCG spec should aim to provide the means for a designer/team to model the relationship of the tokens to modes, but leave the business of how tokens are translated to generator tools. Leveraging the $extensions field to provide instructions for generator tools accomplishes that very well.

It seems prudent to me, that the DTCG spec should aim to provide the means for a designer/team to model the relationship of the tokens to modes, but leave the business of how tokens are translated to generator tools. Leveraging the $extensions field to provide instructions for generator tools accomplishes that very well.

As a translation tool implementer I can safely and surely say that we will never go beyond the specification. If there is an expectation of some behavior in translation tools, it needs to be specified. We don't see the point of building tools for a specification and then having to invent critical implementation details.

I think it is dangerous to consider $extensions as an escape hatch that can be used to fill in holes in this specification. We have zero interest in keeping track of what other tools are dumping in $extensions and building logic around that.

$extensions is only realistically useful to store data to be used by a single tool or multiple tools by a single vendor. It isn't suitable as a basis for complex features that must work in all tools.

I am still hoping that this format looks at the whole picture, design tools and translation tools.

Linking to #187 to group all theming related discussions together.

Hi folks, long time no visit. @kevinmpowell pointed me in the direction of this thread to weigh in.

Note

tl;dr, I recommend different files for different contexts (themes) and I have no idea how the group could create a schema for token aliasing because humans have opinions on what's best for them.

One of the things I'm concerned with here depth that this is introducing, and not a good depth in my opinion. Both in terms of variation and in terms of diving into an object for values. The former is difficult for humans, the latter is challenging to code. Specifically about coding to a complex object spec; diving into trees several levels deep with variations of keys looking for values sounds like an engineering interview question I have in my nightmares.

What I'd like to propose is aligned with the way I believe humans think of token assignment. We aren't thinking about light and dark mode simultaneously, we think of them one at a time. Therefore, the first step to my recommendation is to first focus on solving the theme layer; which I believe should be a nearly flat structure of semantic token to value assignments. No variation of light and dark, because each file is meant to relate to a single context. Here's a very minimal example of what could be considered a "light" theme based on the spec today:

{
  "ux-surface-backgroundColor": {
    "$type": "color",
    "$value": "white"
  }
}

As a human thinking about how I want to assign color semantically, I scope myself to a single context (ie., mode/business-need/experiment) at a time. If I'm working on a light theme, I'm not trying to find all of the light theme values in a single file, I'm working in the file that is meant to convey the light theme. Anything else is just noise. Granted, I recognize that the expectation is to not work with token files directly but I believe there's an opportunity for simplicity here. Defining each theme file as a single context helps focus the curation exercise. Having everything available at once is triggering Hick's Law. Even with tools, this would be visually daunting. Speaking from experience here, if we were to attempt to put all tokens across all brands in the same file at GoDaddy (with numerous reseller brands), we're talking about a number around 24,000 token assignments in a single file or hidden within the depths of nested UI.

Furthermore, this supports the ability to have a scoped theme within a larger page. You can be sure that anything placed within that scope (ie., inverted hero) will be covered either by the page level theme, or the one expected to be applied in scope. There's more about that in this post, where I recommend that variations of "on dark" as token names are not scalable.

What I've mentioned above covers semantic token to value assignment; the tokens that will eventually apply directly to UI elements. What it doesn't cover is additional tokens that I recognize would be helpful for brand organization. In my view this layer is wildly unique among teams, brands, and organizations because it is often a reflection of the personalities of the people who maintain these layers.

I ask myself why people need additional layers all the time. In reality there's nothing stopping someone from just assigning blue as the value to a semantic token. Again, speaking as someone who has been making these kinds of value assignments for years, creating a color.blue.500 token just means I'm copying (or assigning through some tool) the color.blue.500 token over and over across them theme layer in the same place I would be copying blue. In reality color.blue.500 shouldn't change because we don't easily know what this means for the assignments at the theme layer. Seeing the change of color.blue.500 in a sample of UIs doesn't cover what it means for all UI as a semantic tier does.

The only reason I can think of that these other tiers could exist is to be able to speak about the token conversationally. It's clearly more helpful to say color.blue.500 over #1d9bf0, there's even some categorization built into the name. It feels helpful from a human perspective.

Above lies the challenge, as I imagine it'll be impossible to propose avoiding additional tiers. It'll be more of a challenge to define what these tiers look like and support all of the variations the humans may dream up (I can see the marketing team coming in there, wanting a color called make-it-pop-pink).

From there, it's a matter of importing this tree into the theme file:

import colors from './colors.json';

// OR, and probably more desirable for multibrand

import { colors } from './my-brand-styles.json';

// For the "light" theme
const tokens = {
  'ux-action-backgroundColor' : {
    '$type': 'color',
    '$value': colors.blue['500']
  }
}

export default tokens;

I'm admittedly torn having an opinion at the additional tier layers (aka token aliasing). On the one hand as a specification meant to cover tokens, it should have some well-defined schema where systems can share, read, and manipulate maintaining expectations. However, on the other hand, I find these additional tiers mostly useless personally. Which probably answers the question for the group about whether to include them or not. Clearly they should be included for the greater design community, I just can't imagine how we're going to cover everything people could want to do here in a schema without it being a dynamic dumping ground or tied up in ego.

I'll also admit that I lean into the semantic layer hard. I believe that UI designers are really theme authors; people interested in what the values of the UI are. And that UX designers (folks interested in the user experience) could submit wireframes which are wired semantically (this is a button, this is a card) to point to the semantic tokens to be informed by the theme author. This presupposes that UX designers don't have opinions about what color their design should be, and I know that's absolute crazy talk.

I think I'm aligned with @ddamato's feedback here, but I'm wondering if there's some assumptions being made about the token authoring process that are influencing these proposals. I don't want to derail this thread, but I think it's useful to do some context setting.

Should a tokens file be easily edited/maintained by a human in a text editor?

149 is still sitting open...and @romainmenke has posed some follow up questions we've yet to respond to there.

A. If yes - a tokens file should be easily edited/maintained by a human in a text editor

Then I see the issues of interleaved modes/themes within a tokens file as hugely problematic. I believe token authors will be most interested in maintaining how different tokens interact within the same theme, rather than focusing on a single token's value across themes.

B. If no - tokens files will almost never be edited/maintained by a human in a text editor

Then a design tool like Figma or Sketch, or some other token management tool could easily parse the file, and allow me to filter my tokens file down to the context of a single theme, regardless of the underlying structure within the tokens source file.

I think this is an important distinction, and I sense most of the proposals here are assuming B, but I'd love clarification on that from @jjcm @gossi @c1rrus and others.

Multi-file approach

I think @dbanksdesign's explorations and conclusion in this article and @ddamato's examples above are worth considering here.

I could imagine something like this, where the responsibility for naming themes, defining fallbacks, etc is something a token tool handles, but not something that requires that complexity to be added to the token files themselves. To me, the benefit is simpler (and smaller) token files to maintain, organized by context. If a token editor needed to check the different values for a single token across files, a token tool could aid that use case, but I'm assuming that's needed less often than working across tokens within a context.

Feedback

I'm interested to hear arguments against the multi-file approach and what the drawbacks might be. I haven't seen this method discussed in detail in this thread yet.

And great discussion everyone! Really appreciate the community involvement here.

I'm reading through this, but especially on @ddamato's post I came to realize we've been using very overloaded terms and everybody is defining them differently, which in turn makes it complicated for us to properly communicate (as I don't really know what Donnie means with theme or context). I consider myself part of this problem and started to define them for my blog series.

I start with definitions at first, then I have an excerpt for you, that is one of the articles of the series (which is about theming, which I actually started writing before I know about this issue) - but is exactly targeting this issue. And later I propose this into a something formalized that has draft spec character for us to discuss and invent further.

1. Definitions

(Design Token) Feature

Features provide a customization option for users to modify their experience.

Features encode user preferences (e.g. color scheme or color contrast). Design tokens may or may not support a feature.

Theme

Themes contain the design decisions of your brand with differing capabilities (features and scopes) and are exchangeable within your product.

A theme defines its capabilities and delegates implementation of these to its tokens.

(Design) Token Designers

A token designer develops designs tokens. The token design includes references, possible transforms, ambients and computed tokens (this can be done by designers and/or developers !).

(Design) Token Consumer

A user applying a token - most likely designers and engineers, but also includes branding and marketeers or generally speaking people applying the brand by using design tokens.

(Design) Token Users

Token users include token designers and consumers, it's a way to refer to both parties. And includes the fact a token designer here, can be a token consumer there.

Tooling Authors

Folks to support design tokens in their tools and define the processes we use for crafting.

(End)Users/Customers

People like you and me visiting apps and sites and experience products applying design tokens for their theme.

Product

The application, website or service an enduser is using. A product uses a design system.

2. Theming with Design Tokens (an Excerpt)

The definitions above seem a little exaggerated, but I use them in the article, which dives more into the term Theme.

-> Theming with Design Tokens (Draft)

I know people use theme to describe their light or dark color-scheme, which is still possible for them to do. I think, the key is to separate this into theme and features. As this group aims to land a spec that targets enterprise size solution, the more range the definitions cover, the better.

⬇️ ⚠️ From here on, I expect you to have read the excerpt above.

3. Formalizing a Spec Draft

When formalizing this, it is critical to define goals and non-goals.

Goals:

The spec is more like a schema to a database, as the token file is a storage of structured data
The spec is to provide a structure to in which information can be stored
The spec is providing all the information for tools to understand the data much like humans
The spec is human friendly and allows manipulation in text-editors
The spec allows for validating the data to ensure data integrity

Non-Goals:

The spec is not based or constraint by implementation behavior
The spec does not assume or prefer specific working mode (tool vs manual, context or global editing)

With that in mind, I worked on a draft. I'll have to make assumptions on the way (that also target @kevinmpowell's questions).

1. Define the Capabilities of a Theme

Assumption 1: One folder with multiple token files contain a theme.

So far the spec defines the structure within one file. But I can't remember reading about multiple files (please send a link, if there is a mention). That is I assume it the way style-dictionary does: Multiple files in one directory, then all files are merged into one in-memory "database" which is then translated into desired formats.

For that a theme first must mention its capabilities in terms of features and scopes. I'm using typescript here to model this:

interface Theme {
  name: string;
  features?: Features;
  scopes?: string[];
}

interface Features {
  [feature: string]: {
    default: string; // must be available in options
    options: string[];
  }
}

... and an example to it:

{
  "name": "ocean",
  "features": {
    "color-scheme": {
      "default": "light",
      "options": ["light", "dark", "midnight"]
    },
    "color-contrast": {
      "default": "low",
      "options": ["low", "high"]
    }
  },
  "scopes": ["invert"]
}

This information can be read, interpreted and understood by tools and humans. When authoring the theme, these are the constraints.

2. Multiple Token Values

I'd follow the approach for multiple token values, but take in the idea from @ddamato about context oriented editing and evolve from there.

// intents.tokens
{
  "intent-action-base-background": {
    "$type": "color",
    "$value": "darkblue"
  }
}

and another file for dark:

// intents-dark.tokens
{
  "intent-action-base-background": {
    "$type": "color",
    "$value": {
      "$value": "lightblue",
      "color-scheme": "dark"
    }
  }
}

The $value definition gets more detailed by describing the constraints under which this value is valid.

Next up, bring in a scope:

// scope-invert.tokens
{
  "intent-action-base-background": {
    "$type": "color",
    "$value": {
      "$value": "lightblue",
      "$scope": "invert"
    }
  }
}

The same token defined three times, in three different files. They can be compiled into one file:

// compiled.tokens
{
  "intent-action-base-background": {
    "$type": "color",
    "$value": [
      {
        "$value": "darkblue"
      }, 
      {
        "$value": "lightblue",
        "color-scheme": "dark"
      }, 
      {
        "$value": "lightblue",
        "$scope": "invert"
      }
    ]
  }
}

... and both formats are valid and acceptable. Apparently, when all of the files above are merged, the compiled one is the output.

Let's turn this into a formalized definition, using typescript again (simplified of course):

/* 
this is the way how a current value can be defined, depending on its type, etc. 
As this is not relevant here, I assigned it to a string - but give it a name to 
make the connection to the work that was already put into here
*/
type CurrentValue = string;

interface ConstrainedValue {
  $value: CurrentValue;
  $scope?: string; // must be the values defined in meta file
  [feature: string]: string; // must be the values defined in meta file
}

type SingleValue =
  | CurrentValue
  | ConstrainedValue;

type Value = 
  | SingleValue
  | SingleValue[];

This will continue to use special fields prefixed with $ and features use user-defined values. That rule makes it possible to validate the token files.

Validating the Spec

Ok, so here is a good way to validate the spec against some test-cases and whether it confirms ot not.

Is the spec coherent? yes, capabilities can be defined in the meta file
Does the spec allow people to define their situations? yes, capabilities can take any user-defined values
Does the spec allow validation of token files? yes, as token values are defined based on the user-defined values and the structure accounts for them. Validation routines can be written by tooling authors
Does the spec prefer or assume a specific working mode? no, it can be used in a single file or multiple files and authors can define their own organization
Is the spec human friendly? yes, especially as it allows authors to define their own organization of files with their prefered working mode.
no, it may be cumbersome to always define the constraints when working in one file that always uses the same constraint (see open question below)
Is the spec tool friendly? yes, thanks to the meta file. Which contains the information about the capabilities. That's the required information for tools to generate UIs based on these information.
At the same time allows tooling authors to write theme-switchers.
It also allows to take a collection of *.token files and wrap them in a custom build scripts for filtering or put a REST API around them as demonstrated in the article.
Is the spec coupled to a particular implementation? no, the spec defines a structure to organize data into information.
Is the spec coupled to environment implementation details - such as media queries for color schemes? no, that's up to authors to define their feature options and implementation to connect this with the given technical environment.
But there are given technical constraints for color-scheme and color-contrast, how does the spec connect here? I see it similar to how the $type defines the structure of $value. It potentially started with only primitive values (string, number, boolean) into what border-radius needs and by that enhancing the spec with more of such use-cases. I can see the same evolution happening here, in which the spec enhances such types. I would even want to see this, but at first is to create the boundaries and frame the picture in which this can happen.
You like that meta file very much, is that a solution to everything? 😎 haha, no. I'm also not working for meta. I needed a name to refer to that, can also be a theme config 😁

Open Questions

There are a couple of open questions

Q: How to find the "meta" file?

Is there a special name meta.tokens or does each file reference the meta file?

{
  "$reference": "path/to/meta.tokens"
}

Probably, a question asked too early?

Q: What's the conflict resolution algorithm when merging files?

As the spec allows for validating the files against the capabilties set in the meta file, when merging files (especially through tooling) they can validate the files and log error messages, such as:

Found two values for the same token: xx-yy-zz
Missing token value for dark color-scheme and low contrast for: another-token
...

I think, that's already the case for style-dictionary. Tools can provide configuration for severity levels (warn vs error).

Q: Do I always have to set the constraints on each token value? Can't I set it for the entire file?

I can already hear the people scream about this 😇 That's a question for constraining an entire group and not part of this issue. I've seen other discussion around this, please link them.

Q: Do features need to be $features in ConstrainedValue?

With features being next to $value and $scope they eat up all space for future enhancements (unless they come $-prefixed). Which takes away real-estate within the object. Instead they can be fenced in its own object:

intents-dark.tokens
{
  "intent-action-base-background": {
    "$type": "color",
    "$value": {
      "$value": "lightblue",
      "$features": {
        "color-scheme": "dark"
      }
    }
  }
}

that is making it more explicit for sure, but also a bit more cluttering. Tools wouldn't care, but reduces the "human-friendly" part of it.

I don't have an answer but wanted to state I was discussing this situation.

My own opinions on this: I took the comments of this issue matched them with my own research and come up with this draft. I think, it answers a lot of questions. It is despite quite concise and I even managed to keep it human friendly (I hope). I'm somewhat surprise about the outcome here (now that I wrote up all this). Super interested in hearing your feedback.

A few points from @gossi's post.

Shared language is definitely an area of improvement here. I think of a theme as a collection of semantic token and value pairs. Then that theme is like a cassette tape which provides a mood which can be swapped out to provide different moods. In a way, if I'm in a sad mood, it'll be very hard to make happy music. Same goes for theme curation and thinking in terms of the light theme as a complete unit of work apart from other themes. This is where the idea of context comes in. Context is thinking about a certain mood specifically. Context switching happens when I'm trying to think about light and dark near-simultaneously and where mistakes will occur.

For that a theme first must mention its capabilities in terms of features and scopes

This is where I think we're going to have the most trouble. As an example, I'm not sure how this might support changes for locale (eg., black colored buttons do not perform well in China, so we have to support a different colored button in this context)[^1]. There's going to be other considerations here past ones the token designer might maintain. Experimentation comes to mind here which is often either marketing or product driven. This is why I'm not confident that the tokens specification should hold the explicit conditional as a schema. I think of it more as a human-readable field.

// default-zh-CN.json
{
  "$description": "Changes made for the zh-CN market",
  "$tokens": {
    "intent-action-base-background": {}
  }
}

This is because the way that we trigger when these tokens should be used will differ between platforms or organizational resources.

However, if the human-readable field is too freeform, I think the task is then how do we encode the purpose for this specific theme into a well-defined schema? Keeping in mind that it'll be near impossible to identify all the reasons one might want to create a theme. Something like an array of $tags (or another name $categorization, $purpose, $context?) might work here for systems to have their own way of categorization (as the categorization of themes is also highly influenced by the people using them). Requests for a theme would search against all the themes with a given set of tags and find the best match. An enhancement to $tags could set a priority to the field showing that finding the correct market is a higher priority than finding the correct contrast theme.

// zh-CN-high-contrast.json
{
 "$tags": [{
    "$type": "market",
    "$value": "zh-CN",
    "$priority": 1
  }, {
    "$type": "contrast",
    "$value": "high",
    "$priority": 2
  }],
  "$tokens": { ... }
}

You could also adjust this as an object like $tokens so there are tokens that represent the theme categorization itself (how meta).

// zh-CN-high-contrast.json
{
  "$tags": {
    "zh-CN-market": {
      "$type": "market",
      "$value": "zh-CN",
      "$priority": 1
    }
  },
  "$tokens": { ... }
}

Though I'm not sure what the benefit of providing a token-like construction for this would be. As mentioned above, I imagine systems would loop through the $tags to identify the best match and return the $tokens from that.

[^1]: This also opens up the possibility of theme inheritance; the ability to use one theme as a base and update a subset of semantic tokens with new values. In practice I shouldn't need to copy all of the values from the base theme and then change one value for the zh-CN market. We should have a solution (if one doesn't exist yet) to show that one theme can extend from another. Admittedly, having all the values in one file is a solution but I'm hoping for a way which reduces the context. In other words, it's less intimidating to make a zh-CN change in a file with only a few tokens, than one that has all the tokens.

I love this thread, as it really demonstrates the complexity of something that, at the surface, seems intuitive or simple.

Couple of thoughts, and a proposal:

1. There are times where someone will care about and want to use multiple themes at the same time.

Which is to say, it isn't always true that someone works with one theme at a time. A few cases that demonstrate this: a. One part of a page/view uses the "dark" theme, and a different part uses the "light" theme (see below) b. One part of a page/view uses both a "dark" theme, and a "high density" theme, and a "brand X" theme.

For example, at Stripe, we have a few parts of our dashboard that we want to always show using the "dark" theme (some of our developer tools), regardless of the user-selected theme. Likewise, we have parts that always need to be in the "light" theme (previews of branded emails and invoices). That means, on one page, there might be a part of the page which has the user-selected theme, part that is always in a "dark" theme, and part that is always in a "light" theme.

The multi-file way of authoring themes might be convenient when the number of tokens/themes is high, but I think it would be a bit restrictive if it was the only way. We should come up with a robust way of authoring themes in the same file that can be extended to multi-file authoring.

2. Theme metadata like features, tags, priorities, and theme types may be over-complicating the spec

While the proposals by @gossi and @ddamato are very thoughtful and well-articulated, I worry that building too many new features into the spec is shifting complicated functionality from platforms up into the tokens format and into token translation tools themselves.

How interfaces are styled based on contexts — user-selected theme, accessibility preferences, localization, etc — varies widely from platform to platform. CSS, JS, Swift, Objective-C, Java, and Kotlin each have their own capabilities, quirks, and best practices for contextual styling. I think we should leave the complexity of understanding user contexts and applying styles to the UI to those platforms.

The token spec can (and I think, should) provide a high-level map that can be used by the platforms, but to make interoperability easy, it should be as lightweight as possible.

Proposal: let's define themes with the least modifications to the current spec

Starting with the minimum valid token file:

{
  "color-brand": {
    "$value": "#ff0000",
    "$type": "color"
  }
}

We’ll make only one change to the spec:

We introduce a "$themes" key.[^1] The value of $themes should be an object consisting of key-value pairs. the keys should be the name of the theme. The value should be either: A tokens object A string containing a path to a valid tokens.json file

So, a token file with themes will look like this:

{
  "color-brand": {
    "$value": "#ff0000",
    "$type": "color"
  },
  "$themes": {
    "dark": {
      "$description": "the dark theme",
      "color-brand": {
        "$value": "#0000ff",
        "$type": "color"
      }
    }
  }
}

A token translator can apply the theme by merging the values in the $themes object with the "main" json object. Let's look at some more realistic examples:

Example: a dark theme and light theme with token aliases in a single file

{
  "colors": {
    "white": {
      "$type": "color",
      "$value": "#ffffff"
    },
    "black": {
      "$type": "color",
      "$value": "#000000"
    }
  },
  "$themes": {
    "light": {
      "foreground": {
        "$value": "{color.black}"
      },
      "background": {
        "$value": "{color.white}"
      }
    },
    "dark": {
      "foreground": {
        "$value": "{color.white}"
      },
      "background": {
        "$value": "{color.black}"
      }
    }
  }
}

Merging the light theme object with the main tokens object, we get the resulting json:

{
  "colors": {
    "white": {
      "$type": "color",
      "$value": "#ffffff"
    },
    "black": {
      "$type": "color",
      "$value": "#000000"
    },
    "foreground": {
      "$value": "{color.black}"
    },
    "background": {
      "$value": "{color.white}"
    }
  }
}

Which is a valid, parsable tokens object! The translation tool is free to do with this as it pleases.

Example: the same themes with multiple files

tokens.json

{
  "colors": {
    "white": {
      "$type": "color",
      "$value": "#ffffff"
    },
    "black": {
      "$type": "color",
      "$value": "#000000"
    }
  },
  "$themes": {
    "light": "path/to/lightTheme.tokens",
    "dark": "path/to/darkTheme.tokens"
  }
}

path/to/lightTheme.json

{
  "foreground": {
    "$value": "{color.black}"
  },
  "background": {
    "$value": "{color.white}"
  }
}

path/to/darkTheme.json

{
  "foreground": {
    "$value": "{color.white}"
  },
  "background": {
    "$value": "{color.black}"
  }
}

The token translator could follow the path to the tokens file, and do the merge as before. This would result in the same outcome as above.

Example: creating a “patch”

There may be cases in which a user doesn’t want to output full theme. Perhaps they only want a “patch”, which includes only the tokens in the theme file, fully resolved.

Specifying a patch as an output, we could imagine a translator only resolving the token aliases and outputting the tokens specified in the theme. Using our tokens specified above, that might look like:

Light theme patch

{
  "colors": {
    "foreground": {
      "$type": "color",
      "$value": "#ffffff",
    },
    "background": {
      "$type": "color",
      "$value": "#000000"
    }
  }
}

And again, the token translator could parse this into a useful output. This might be something that is put in a media query, where you don’t want to re-define every token.

Example: stacking themes

As other folks have mentioned, you may want to apply multiple themes at a time. This could be done by the translation tool, in the order specified by the user at compilation time.

In this example, I’ll apply a dark mode theme on top of a brand theme.

Here’s the tokens file:

{
  "colors": {
    "blue": {
      "200": {
        "$type": "color",
        "$value": "#75D5E8"
      },
      "700": {
        "$type": "color",
        "$value": "#04438C"
      }
    },
    "red": {
      "200": {
        "$type": "color",
        "$value": "#75D5E8"
      },
      "700": {
        "$type": "color",
        "$value": "#890D37"
      }
    }
  },
  "$themes": {
    "blueBarracuda": {
      "color": {
        "brand": {
          "200": {
            "$value": "{color.blue.200}"
          },
          "700": {
            "$value": "{color.blue.700}"
          }
        }
      }
    },
    "redJaguar": {
      "color": {
        "brand": {
          "200": {
            "$value": "{color.red.200}"
          },
          "700": {
            "$value": "{color.red.700}"
          }
        }
      }
    },
    "light": {
      "button": {
        "foreground": {
          "$value": "{color.brand.200}"
        },
        "background": {
          "$value": "{color.brand.700}"
        }
      }
    },
    "dark": {
      "button": {
        "foreground": {
          "$value": "{color.brand.700}"
        },
        "background": {
          "$value": "{color.brand.200}"
        }
      }
    }
  }
}

If we apply the blueBaracuda theme, and then the dark theme, we get the following output:

{
  "colors": {
    "blue": {
      "200": {
        "$type": "color",
        "$value": "#75D5E8"
      },
      "700": {
        "$type": "color",
        "$value": "#04438C"
      }
    },
    "red": {
      "200": {
        "$type": "color",
        "$value": "#75D5E8"
      },
      "700": {
        "$type": "color",
        "$value": "#890D37"
      }
    },
    "brand": {
      "200": {
        "$value": "{color.blue.200}"
      },
      "700": {
        "$value": "{color.blue.700}"
      }
    }
  },
  "button": {
    "foreground": {
      "$value": "{color.brand.700}"
    },
    "background": {
      "$value": "{color.brand.200}"
    }
  }
}

The bonus to this approach is that I do not need to define separate dark and light themes for each brand! The token aliasing takes care of that for me. In this way, I think we can avoid the combinatorial explosion of multiple composable themes.

Example: defining themes on the token level

I believe this approach works regardless of if you define a theme at the "base" level, or if you define it at a group level, or even a token level.

{
  "colors": {
    "blue": {
      "$type": "color",
      "$value": "#75D5E8",
      "$themes": {
        "dark": {
          "$value": "#04438C"
        }
      }
    }
  }
}

Applying a theme simply requires merging the selected theme up into the token above it. In this case, applying the dark theme yields:

{
  "colors": {
    "blue": {
      "$type": "color",
      "$value": "#04438C",
    }
  }
}

And parsing can continue from there.

The downside to this approach

Because we’re not specifying the order that themes are applied within the tokens file itself, analyzing the file will not tell you if a theme’s aliases will resolve or not. For instance, if I took my brand-and-color-theme example above and only applied the dark theme (skipping the brand theme), the final output would have an alias to a {color.brand.200} token that does not exist. However, this can be caught and handled like any missing reference, and wouldn’t need special treatment just because it’s a theme.

I think that the tradeoff of flexibility, composability, and simplicity is worth the possibility of this kind of error in the transation pipeline.

Ok, so how do translators know how to handle contexts?

We can offload the complexity of specifying which theme goes with which context to the translator.

Styledictionary, for example, separates out the tokens definition from the configuration for processing the tokens. Therefore, we could expect to have the themes defined as above, and have the instructions for processing the themes done however instructions are provided to the translator (a configuration file, flags on a command line script, etc): theme A should be translated into a (prefers-color-scheme: dark) media query, theme B should be put into a separate .css file, etc.

This gets at an as-of-yet-unexplored conversation about how much a .tokens file should be providing documentation of tokens vs. providing instructions to a translation tool. I expect a hardy and spirited discussion to follow :)

[^1]: this doesn’t have to be “themes.” It could be “modes,” too. I actually would prefer “contexts” to these, as it is is both more general and thus more applicable.

@ilikescience How is this different form regular groups or using multiple files?

We can offload the complexity of specifying which theme goes with which context to the translator.

Styledictionary, for example, separates out the tokens definition from the configuration for processing the tokens.

Can you provide examples?

As an implementer I am not very interested in providing complicated configuration knobs and dials so that users can teach my tool that $theme.dark should be rendered with @media (prefers-color-scheme: dark).

For us it is much more interesting to have pre-defined conditionals for cross platform features like native dark mode. And then having some syntax feature in the specification that connects a token value with that native feature.

As I said before, these types of "native conditonals" are different from "theming" or "branding".

I think native conditionals need to be part of the specification and must be fully described, no hand wavy "the translation tools will fix it", but strictly defined :)

I think it is critical to not confuse native conditional features with theming or branding. From a designers point of view these might seem similar but they aren't the same.

I'm not sure I clearly understand the issues of tool developers. I'm looking at it from the designer's point of view.

Theme (or "mode") is an list of resources which should be used to render the UI ("resources" I mean are JSON files in a specific format). If this is the case, all we have to do is standardize the format of the configuration file.

We will need to describe things like:

Theme name
List of necessary resources for rendering the theme
Some pointers to what values to count as defaults

Also note – dark beer and light beer can be described using the same BeerJSON recipe format - there is no need for additional abstractions 😄🍺

{
    "defaultTheme": "light",
    "themes": [
        {
            "name": "light",
            "themeResources": {
                "set1": "/path/to/tokens.json",
                "set2": "/path/to/tokens.json",
                "set3": "/path/to/tokens.json"
            }
        },
        {
            "name": "dark",
            "themeResources": {
                "set1": "/path/to/tokens.json",
                "set2": "/path/to/tokens.json",
                "set3": "/path/to/tokens.json"
            }
        }
    ]
}

One way to put it :

theming or branding token sets are applied by developers.
native conditionals like dark mode are applied through device aspects or user preferences.

Yes it is technically possible to have developers write all the glue code for native conditionals to work but we don't see the benefit to that. If a designer already defines the relation between values why should a developer do that work again and again? Is it not the purpose of this specification to enhance the experience of everyone by having a format that communicates design decisions?

Also keep in mind that a "dark design" is not "dark mode". Native dark mode is a feature through which end users communicate that they want a darker rendering, a "dark design" is a set of design choices.

By throwing "dark mode" together with "theming" or "branding" we are oversimplifying the issue at hand. It is important to consider it fully.

1. There are times where someone will care about and want to use multiple themes at the same time.

Which is to say, it isn't always true that someone works with one theme at a time. A few cases that demonstrate this: a. One part of a page/view uses the "dark" theme, and a different part uses the "light" theme (see below) b. One part of a page/view uses both a "dark" theme, and a "high density" theme, and a "brand X" theme.

It seems I need to be more clear in the distinction I'm trying to make. Yes, we want to use multiple themes at the same time but I don't think anyone wants to curate multiple themes at the same time. In order to create a light theme, I need to put my mind into the context that everything in this area is colored in "some light way". In any of these proposals, we can still apply multiple themes.

2. Theme metadata like features, tags, priorities, and theme types may be over-complicating the spec

While the proposals by @gossi and @ddamato are very thoughtful and well-articulated, I worry that building too many new features into the spec is shifting complicated functionality from platforms up into the tokens format and into token translation tools themselves.

No matter what, the end of this thread will be suggesting a new feature.

How interfaces are styled based on contexts — user-selected theme, accessibility preferences, localization, etc — varies widely from platform to platform. CSS, JS, Swift, Objective-C, Java, and Kotlin each have their own capabilities, quirks, and best practices for contextual styling. I think we should leave the complexity of understanding user contexts and applying styles to the UI to those platforms.

While I agree that the execution of the tokens against the UI will differ, I argue that the intentions do not. Declaring some part of the UI to be the "button background color" is persistent across interfaces, the way in which that is assigned is determined by the platform. What I recommend is that the tokens file describes intentions and the platforms ingesting these tokens know enough about their own infrastructures to make necessary transforms from there.

The token spec can (and I think, should) provide a high-level map that can be used by the platforms, but to make interoperability easy, it should be as lightweight as possible.

Agree, except:

Example: a dark theme and light theme with token aliases in a single file

{
  "colors": {
    "white": {
      "$type": "color",
      "$value": "#ffffff"
    },
    "black": {
      "$type": "color",
      "$value": "#000000"
    }
  },
  "$themes": {
    "light": {
      "foreground": {
        "$value": "{color.black}"
      },
      "background": {
        "$value": "{color.white}"
      }
    },
    "dark": {
      "foreground": {
        "$value": "{color.white}"
      },
      "background": {
        "$value": "{color.black}"
      }
    }
  }
}

This single file is going to become unwieldy with the number of tokens and brands that some organizations possess. It seems fine for light and dark, but we've already had examples of other modes, markets, and experiments that simply should not coexist because their purposes are fundamentally different.

Returning to the basis behind my recommendation. Splitting each file by purpose focuses the curation exercise to the reason why this specific collection of token-value assignments needs to exist. This reduces the problem down to the one I mentioned above:

How do we encode the purpose for this specific theme into a well-defined schema?

We can work with the spec as-is without defining this moving forward and still support theming. Systems need to request the appropriate file which represents the theme. What I proposed earlier is an attempt to standardize how a system might determine which collection of token-value pairs to deliver. Since having an arbitrary key (ie. light) means a system must specifically look for this one theme versus a system looking for a best match. Again, not important to v1 of theming.

Thanks for drafting up such an elaborate proposal @jjcm , I quite like it.

apologies in advance for interchangeably using the terms "theme" and "mode"

My main concern is also what @nesquarx mentioned, about scalability. If I'm adding a theme with only a small bit of overrides, after the fact.. it might be easier to have this in a separate file, so I can easily see what a theme overrides on the core.

@connorjsmith 's comment addresses my concern in that regard:

Depends on the heuristic we'd want to use for what constitutes an "extension" of the base theme (either via some sort of $extends, or just via name matching), but I'd think that defining more modes would be something like this

I also think that is very closely related to my proposal of an $extends property, some sort of simple metadata that defines a contract between token files or token groups even.

I share @romainmenke 's sentiment (to a small extent) that we should consider how the token format is processed and then translated to various platforms. For example, for Web (CSS), there's multiple ways you might prefer your output.

:root {
  --some-color: #FF0000;
  --some-unthemed-color: #000000;
}

:root[mode="green"] {
  --some-color: #00FF00;
}

But, you could also have them as separate stylesheet files, and import/use the right one based on the current mode. You could even decide that you want your core tokens, even if the current theme has no override for it (so you're using the fallback then), included in the themed file:

default.css:

:root {
  --some-color: #FF0000;
  --some-unthemed-color: #000000;
}

green.css:

:root {
  --some-color: #00FF00;
  --some-unthemed-color: #000000;
}

So above here, even though there is no specification for mode green for the unthemed token, we still get all the tokens in the output.

I think the choice between single-file, multi-file, and also the choice between full output vs overrides only, is always going to depend on (among others):

the platform (web, ios, android, etc.)
the stack (JS-in-CSS, less, CSS, sass, stylus, css modules)
the translation tool (e.g. style-dictionary) and its capabilities for exporting to various outputs

Overall, I think the specification is definitely in the right direction if scalability and flexibility with regards to translating to outputs are considered, which @connorjsmith definitely eased my mind a bit on with the $extends idea.

Hey, just wanted to chime in here as well.

I really love the idea of being able to extend other files. GitHub was already mentioned before as we have a somewhat complex system with 9 themes.

The way we currently have them set up is like this:

1 dark default theme and 1 light default theme
- a dark dimmed theme that inherits from dark default and overrides a lot of tokens
- a dark high contrast and a light high contrast both of which inherit from the default themes and override a good amount of tokens
- a tritanopia and a deuteranopia / protanopia theme for light and dark mode. Those are identical to the defaults and only change tokens that rely on specific colors like green or red

We have out tokens split up into many files including:

all the overrides mentioned above in a file per theme
base color scales are in a specific file
component tokens have their own file
and some more

Having all of them in one file would be a nightmare to maintain, so it is important to be able to use files and merge them together in an individual way.

I also want to warn about the idea that design token files are created by software. I know some folks think this but in reality, this is often not true. First of all, the software is not there yet. Often there are some limits that mean that we need to at least edit files by hand later on. Second, even if software would be able to create all tokens in a way we need it, people will have to approve PRs to token files. They are a very crucial part of the software we build after all. Looking through a giant file with multiple themes makes PRs harder to approve.

TL;DR

files must be easy to edit by hand (it still happens a lot)
users should be able to define themes and tokens across multiple files and merge them later
- this includes having one file per theme
- also includes having a file for component tokens and one for base scales

Having had insightful conversations with @romainmenke (thanks for your persistence ;-) ) and implemented a custom Style Dictionary based token stack, with dark mode, brand themes, locale-specific typography and responsive size tokens for Web, iOS and Android:

I agree with @mkeftz that we need multiple fallbacks, that makes life much easier when formatting the output for platforms that cannot merge their resource files, but only point to a single file at a time where we need each token for each mode / condition to be valued
There are as many good ways to format modes as platforms out there, we've had to
- write a single file with custom selectors/conditionals to express modes (Tailwind, iOS Swift)
- write entire folder architectures where the platform decides in which order to apply them (Android's baked in support for dark mode and locale specific resources)
- write folder architectures with custom platform code to hint in which order to use theme (Android's theme infrastructure)
- write individual tokens to a file, with the value for each mode within every file (iOS xcassets)

I've started a PoC feature to implement multiple values per token, which could both support Romain's proposal and themes (we use it for both) because it wasn't tractable for us to have the output format for modes/themes/conditional values imposed by tool restrictions, it was too platform-dependant.

I would have found it helpful to have predefined conditional values that tools know how to transform (e.g. dark mode to CSS), though I acknowledge we've had to rewrite every transform and format and ended up using almost none of Style Dictionary's built-in code for this complex use case, so I am not 100% sure everyone would agree on how these predefined conditionals should be done. For me, built-in behaviours here would benefit from being overridable though I now agree with Romain they are useful.

Regardless, the same/similar format can be used for both themes and conditional values with a defined semantic. Looking back on my Style Dictionary PoC, having both would cost the same to write as having either. See https://github.com/amzn/style-dictionary/issues/909 if you're curious, though be aware I won't be making progress soon on this due to other commitments.

My main concern is also what @nesquarx mentioned, about scalability. If I'm adding a theme with only a small bit of overrides, after the fact.. it might be easier to have this in a separate file, so I can easily see what a theme overrides on the core.

@connorjsmith 's comment addresses my concern in that regard:

Depends on the heuristic we'd want to use for what constitutes an "extension" of the base theme (either via some sort of $extends, or just via name matching), but I'd think that defining more modes would be something like this

I also think that is very closely related to my proposal of an $extends property, some sort of simple metadata that defines a contract between token files or token groups even.

For us, having distinct files would be unmanageable. We define hundreds and hundreds of tokens, currently with 18 different modes applied. Getting the full picture of what a token's value can be would be difficult, and we find that once the token system is set up, we update tokens rather than updating modes. Having a single file where a token showcases all its values make that easier to manage.

I'd love to see both formats supported by tools, though I believe what tools like Style Dictionary currently do (multiple files with a value per token per file) is too inconvenient for long-term maintenance workflows.

TL;DR

We dont think the $modes example is scalable. We're showing a counter-proposal that extracts the theme definitions outside of the actual token files using a concept we call "Resolvers". This concept allows multidimensionality and granular controls over the final output.

The resolver supports multiple dimensions for token resolution and can handle an arbitrary number of dimensions. It keeps the token specification simple and externalizes the logic for handling additional dimensions outside of the token files themselves.

The proposed resolver introduces the concept of modifiers, which act as "libraries" that are referenced by the source set and can be used to define different dimensions for token resolution. Modifiers can have different values, and their selection is dependent on the input. The resolver also supports the use of contexts to pass values as inputs for context-sensitive design.

Overall, the proposed resolver offers a more flexible and scalable approach to multidimensional token resolution in design systems. It simplifies the token specification by keeping theme definitions outside of it, improves performance, and enables context-sensitive design.

Take a look at our POC site which shows the tokens in action and provides a few examples that can be played with.

This post is a snapshot of our public proposal for resolvers in Tokens Studio available here

We will take all the feedback we receive here to evolve the spec according to requirements

Parameters

It's important to setup some parameters to tailor the discussion of a proposal around:

As @lukasoppermann mentions, files should still be simple enough to be edited by hand. Tooling as it currently stands has not reached the point of maturity where the system is fully automated. Additionally for a new spec it should be simple enough for new users to dip their toes in and test the viability of the solution.
Scalability. This proposal is currently addressing adding support for modes and themes which are two additional dimensions for token resolution. In a real world case for larger systems there might be significantly more dimensions, such as the brand, surface, etc, so whatever solution we provide should be able to handle an arbitrary amount of dimensions.
Simplicity. Referring back to point 1, we don't want to complicate the existing spec for tokens. If we do we end up polluting the spec with responsibilities outside of its core intent of just providing a simple and extensible way to define tokens. Thus we should externalize the logic concerned with handling these additional dimensions outside of the actual token files themselves.

Using the above parameters, the proposal is that we create standalone resolver files which represent the different ways in which a token can be resolved.

Background

This approach is based off the existing work in the Tokens Studio Plugin which currently supports multidimensional tokens through the use of its $themes.json file that it exports with the following shape

[
  {
    //An arbitrary generated id as a hex string
    "id": "12323422b00f1594532b34551306745622567a",
    "name": "brand",
    "selectedTokenSets": {
      // A set that is used to resolve references
      "tokens/core": "source",
      // A set expected to be used in the output
      "tokens/semantic": "enabled"
    }
  },
]

Using this, they are able to support multiple dimensions by using token sets as sources, which means they are able to be referenced for aliases, but do not contribute to the final token values and enabled sets which are then flattened in order to get the final values of the resulting token set once all transformations have been applied.

An example of this in action would be the following where the brand X has different combinations of themes by selecting a combination of differents sets to form that multidimensional theme :

[
  {
    //An arbitrary generated id as a hex string
    "id": "2f440c32b00f1594532bf5b051306724e22136a",
    "name": "Brand X | Light theme",
    "selectedTokenSets": {
      "brand/x/foundation/color/light": "source",
      "foundation/color/appearance": "source",
      "foundation/dimensions": "source",
      "mode/light": "enabled",
      "semantic/actions": "enabled",
    }
  },
  {
    "id": "aa7e80632359bcfc4e09761f8d8f235d02eb41d7",
    "name": "Brand X | Dark theme",
    "selectedTokenSets": {
      "brand/x/foundation/color/dark": "source",
      "foundation/color/appearance": "source",
      "foundation/dimensions": "source",
      "mode/dark": "enabled",
      "semantic/actions": "enabled",
    }
  }
]

The token sets used are typically larger than what is shown and there are cases where there are 40+ sets

There are a number of problems to this approach though. This is currently being applied to create a single addressable space for tokens and is not granular. If you wanted to create a much smaller set that represented each component for example, you could not as the output of the set resolution is a single named token set that is then used to reference the tokens.

Button-Breakdown

The references to the tokens used in this styling are tied to a single addressable token space

Whilst you could add more token sets as enabled if you wanted a button component by adding comp/button under comp/body for example, unless you are following strict naming, your button set might clash with other named tokens in the final generated token set. It is very likely that you would want to define dimensionality on a per component basis if your design system were evolving as well, so that new designers who were onboarding were not overwhelmed by the existing complexity of the system and can focus on isolated components.

In this we find the next property for the resolver that we would need to specify which is specificity. Different components might want to resolve in an isolated manner with different modifiers that might not be global

@jjcm mentions this as well. Modes are based on how the sets want to be consumed by a user, and are not a property of the set themselves, hence why it also makes sense to use standalone files outside of the token spec to define these and apply them independently.

{
      /*
      * Optional name of a resolver
      */
      "name": "Preset resolver",
      /*
      * Optional description
      */
      "description": "This handles the preset from Figma tokens",
      /*
      * A series of sets. The values of the tokens within these sets will
      correspond with the names of the outputted tokens.

      The order of these tokens is important. If keys for tokens are defined within them, the last token will be
      the effective value used
      */
     "sets": [{
          // An optional override of the name of the set. Can be used when tracing the resolution logic or when using `include` modifiers. Read further to see an include modifier in action
          "name": "first",
          // A reference to the tokens. This could vary depending on whether the resolution is occuring through the file system or in memory
          // In this example we assume through the file system through a relative file called core.json
          "values": ["foundation.json"]
        },
        {
            "values": ["semantic.json"]
        },
        {
            "values": ["button.json"]
        },
      ],
      /*
      * These modifiers act as "libraries" that are "imported" 
         and referencable by themselves and the source set
      */
      "modifiers": [
        {
          "name": "theme",
          //Default value of the modifier
          "default": "light",
            //Optional parameter to rename the set prior to resolution
          "alias":"theme",
          //Identifies the modifier type. In this case it is an enumerated value with named key value pairs
          "type":"enumerated",
          "values": [
            {
              "name": "light",
              "values": ["light.json"]
            },
            {
              "name": "dark",
              "values": ["dark.json"]
            }
          ]
        },
        {
          "name": "core",
          // Potential optional parameter to hide this modifier in software that visualizes the resolver
          "hidden": true,
          "default": "core",
          "type":"enumerated",
          "values": [
            {
              "name":"core",
              "values": ["core.json"]
            }
          ]
        }
      ]
    }

Note in the above example json specification since we do not allow arbitrary values, we have left out using the $ prefix on properties as is used in the token spec.

For visual thinkers the following is in effect

resolvers

For the type property there are multiple possible values

enumerated. This is a switch where the selection of the value is dependent on the input. If no value is provided, the default case is chosen. If no default is provided, no set is included.
include. This uses the same approach as the enumerated type however if an input matches the include, then it will replace the set with the same name as in the sets array. If no match occurs, then it is replaced with an empty set

Example

```json5 { //... "sets": [{ //This will be populated by the modifier "name": "theme", "values": [] }, ], "modifiers": [ { "name": "theme", "default": "light", "type":"include", "values": [ { "name":"light", "values": ["themes/light.json"] }, { "name":"dark", "values": ["themes/dark.json"] } ] } ] }; ```

Modifiers can have the same name attribute set. This helps to support more complex cases than just a simple switch. Since names being shared results in the same input property being read, multiple effects can occur due to the same input.

Motivation

Why this approach over the current `$themes` approach?

Having named enumerated possibilities indicates to documentation systems the different possible dimensions. Eg a button resolver could be identified as supporting 3 different brands, 2 modes and 4 themes for example. This is something that cannot be inferred currently. As @romainmenke mentions all possible permutations must able to be detected for code generation tools. By explicitly defining the possible values this should go aways to helping with this.

A point to mention alongside this is the detection of orthogonality surrounding the modifiers. Certain modifiers might override all others, eg disabled which might be a boolean. This might not potentially interact with any other modifier and as such is orthogonal to them and does not result in a value in the permutation matrix

Lower cognitive load. As mentioned keeping all of this within a single global namespace with $themes means a designer needs to keep track of all the different tokens that are currently being used. If they could instead focus purely on logic related to a component, surface, etc by reducing the scope of tokens involved they can intuit the relations between tokens and sets easier.
Faster resolution. For enterprise systems that have potentially thousands of tokens with multiple combinations, getting the final values becomes very slow assuming O(n) resolution time multiplied by O(m) for each dimension. By reducing the amount of tokens in scope, this should be significantly faster. Also since the relationships for which sets consume which other sets are within the resolver, a system could also know which resolvers to cache and which to re-evaluate
Context sensitive design. For cases where context sensitive design is needed we could use contexts to pass through values as inputs to the resolver. An example would be surface logic. A button might change its values in response to the surface its placed on. In a Figma environment this could be implemented by storing a context object at different points of the document tree and then merging it together with a component input to react to whatever it is being placed into. Multiple different points of context could exist within the tree. For example the top level node might set the brand, while a frame within it might set the theme, and within it the surface. These values could be merged together in addition to say a buttons input to determine the final tokens that are in scope

Why is this preferred over embedding `$modes`?

As mentioned before, a single dimension is not sufficient. Even if we were to add $themes in addition to a $mode it would be a matter of time until a design system needed to support additional logic or logic that depends on other values.

Pros

The spec for a resolver can still be kept fairly simple and should still be possible to be handwritten by a human.
We can support an arbitrary amount of modifiers as well as keep the execution time quick even if permutating all possible values by keeping the amount of tokens in scope small by making resolvers that operate only on specific sets. @gossi mentions that larger design systems will likely end up with permutations around ~144.
The token spec does not need to be extended as we are not embedding the resolution logic with the data.
By removing fallbacks from the proposal we don't have to worry about edge cases like @jjcm mentioned with the following.

{
  "$name": "intent-action-base-background",
  "$value": [
    {
      "$value": "white"
    },
    {
      "$value": "darkgrey",
      "color-scheme": "dark"
    },
    {
      "$value": "black",
      "color-scheme": "dark",
      "color-contrast": "high"
    }
  ]
}

const token = tokens.find(t => t.name === 'intent-action-base-background');
const value = token.value.find({ 
  colorContrast: 'high' 
});

As this should be an error if token references are not resolved fully during the resolution.

Resolution logic

The resolution logic is fairly simple. Let's assume that a resolution request comes into a tool programmed in javascript


//Any acceptable resolver definition
const resolver = //...

const inputs :{
    theme:'dark'
    //Additional inputs
}

//async as sourcing tokens from arbitrary storage locations is likely not synchronous 
await resolve(resolver, inputs);

The resolve function would perform a number of steps

Validate the inputs. Input types such as the enumerated type have specific named values that are acceptable. If these are not matched by the provided input, throw an error and fail.
Read the input sets and flatten them into a single new set
Select the appropriate modifiers and read the token files
Perform optional aliasing on the modifier sets if required
Replace input sets if using include modifiers
Flatten the modifiers into a single new set
Resolve the aliases of the input set using values from the modifier set if a value is not found in the input set.

The output of the resolution is a new token set. This token set can then be consumed by any export tools if need be as it is just a simple token set.

Example calculation

We want to resolve the provided resolver:

Resolver

```json5 { "sets": [ { "values": ["foundation.json","semantic.json","button.json"] }, ], "modifiers": [ { "name": "theme", "default": "light", "alias":"theme", "type":"enumerated", "values": [ { "name": "light", "values": ["light.json"] }, { "name": "dark", "values": ["dark.json"] } ] } ] } ```

with the input

{
    "theme":"light"
}

We first validate that there is a modifier called theme that has an acceptable input value of light, then load the necessary sets.

Let's assume the following tokens structures for the example

foundation.json

{
    "gray": {
        "$value": "coolgray",
        "$type": "color"
    },
    "padding": {
        "$value": "4px",
        "$type": "dimension"
    }
}

semantic.json

{
    "primary":{
        "$value": "{theme.accent}"
    }
}

button.json

{
    "padding": {
        "$value": "8px",
        "$type": "dimension"
    }
}

We first squash all the above tokens into a single set that will be resolved with the modifier

Name	Set	Value	Overriden
gray	foundation	coolgray
padding	foundation	4px	✓
primary	semantic	{theme.accent}
padding	button	8px

This results in :

{
    "gray": {
        "$value": "coolgray",
        "$type": "color"
    },
    "primary": {
        "$value": "{theme.accent}"
    },
    "padding": {
        "$value": "8px",
        "$type": "dimension"
    }
}

Now the modifiers are :

light.json

{
    "accent": {
        "$value": "lightblue", // for our light theme we want a lighter shade of blue
        "$type": "color"
    }
}

dark.json

{
    "accent": {
        "$value": "darkblue", // for our dark theme we want a darker shade of blue
        "$type": "color"
    }
}

Note in actual system that implement resolver logic we would not need to load both light and dark json files, only what is specified in the modifier. We show both here for illustrative purposes.

Assuming light was picked as the theme, we would first alias the light set using theme and flatten the values, resulting in :

{
    //These tokens are referenced by the input values and thus we will see them reflect in the output
    "theme":{
        "accent":{
            "$value": "lightblue", // {theme.accent} = lightblue in light.json
            "$type":"color"
        }
    }
    // If there are additional tokens defined here that are not referenced by input values they will not be part of the output
}

Now resolution would occur. We iterate through the tokens within our input set till we find any that require reference resolution and resolve them first using any values found in the input set, falling back to the modifier set as necessary. In this case primary contains a reference to theme.accent. The key theme.accent does not exist within the input set so we look at the modifier set, and find it. We then perform a replacement within the input set resulting in :

{
    "primary": {
        "$value": "lightblue",
        "$type": "color"
    },
    "gray": {
        "$value": "coolgray",
        "$type": "color"
    },
    "padding":{
        "$value": "8px",
        "$type": "dimension"
    }
}

Note if theme.accent had itself been a reference, we would recursively resolve the reference using the same logic.

Usage with export tools

For large design systems that might have multiple brands, the resolution of their tokens as they are finally consumed by the frontend code might be permuting multiple modifiers whilst holding other constant. Eg in an example webapp, the brand foo is known up front and will be the only such value, however the app supports multiple themes and modes.

Either multiple resolvers could be defined on a per brand level or the value of the brand in a single resolver could be held constant whilst evaluating the combinations of the other modifiers.

This would likely be an architectural descision depending on whether the brands have anything in common or not or otherwise require seperate resolvers for governance reasons.

@connorjsmith makes a valid point of the final tools such as style-dictionary deciding the final form of the tokens as a list of css variables, as well as optimizing the final form. Two resolved sets, with light and dark mode set respectively for example, could then be analyzed to optimize the final form of the tokens and removing redundant values .

Resolution aliasing

Consider the following set called size.json.

{
  "sm": {
    "$value": "1px",
    "$type": "dimension"
  },
  "lg": {
    "$value": "10px",
    "$type": "dimension"
  }
}

Let us assume that we want this file to be namespaced so that we can reference this in one of the sets to be resolved. Altering the file directly is not a good solution as it might introduce naming brittleness, as well as potentially have a number of other restrictions like being read-only, owned by someone else, etc.

This also speaks to an assumption we have never mentioned before. It is being assumed right now that everyone is in complete control of their sets, but if we want to support someone referencing another token set(s) eg Material, they should be able to just reference the values without having to modify them, similar to how we import modules in programming.

Rather we could dynamically namespace this when its loaded into the system through an alias like foo to result in

{
    "foo":{
        "sm": {
            "$value": "1px",
            "$type": "dimension"
        },
         "lg": {
            "$value": "10px",
            "$type": "dimension"
        }
    }
}

This allows us to consume other peoples token libraries without directly modifying their files.

Real world use case

The GitHub Primer token sets can be used as an example of applying the resolver to a large system.

The theme specifier here shows a case where we have two dimensions, the light and dark mode, in tandem with a visual impairment dimension that supports:

Light

General
Tritanopia
Standard Colorblindness
High contrast

Dark

General
Dimmed
Tritanopia
Colorblind
High contrast

Note the addition of dimmed. This will be relevant to show where architectural choices might affect how resolvers are used.

Starting with the light mode, we see that there are common sets purely used to reference that should not be used for the output.

These are the src/tokens/base/color/light/light.json5 set and the src/tokens/base/color/light/light.high-contrast.json5 which is used for light-high-contrast. Since the src/tokens/base/color/light/light.json5 is used for all values, this could be set as a include in the modifiers

Note In this case we are assuming that this resolver exists in the src/tokens folder

Resolver

```json { "sets":[{ //The current solution expects to get actual values from the light and dark base sets as the output. As this is dynamic depending on the dimension. We include an empty set here which will be overriden by the modifier "name":"theme", "values": [] }], "modifiers":[ //This is a common set that is used for all resolutions, but is not expected in the output so we use it purely for context { "name":"base", "type":"enumerated", "default":"common", "values": [ { "name":"common", "values": ["src/tokens/functional/color/scales.json5"] } ] }, { "name":"theme", "type":"include", "default":"light", "values": [ { "name":"light", "values": ["base/color/light/light.json5"] }, { "name":"dark", "values": ["base/color/dark/dark.json5"] } ] }, //An example of a repeated modifier. The above performs a different action to the the below defined modifier with this providing a context source to read from { "name":"theme", "type":"enumerated", "default":"light", "values": [ { "name":"light", "values": ["src/tokens/functional/shadow/light.json5`,`src/tokens/functional/border/light.json5"] }, { "name":"dark", "values": ["src/tokens/functional/shadow/dark.json5`,`src/tokens/functional/border/dark.json5"] } ] }, //etc, we add visual modifiers for each of the common visual modifiers ] } ```

Since the high contrast set is included, but only if it is using this modifier, we use a include modifier

Updated resolver

```diff { "modifiers":[ //... { "name":"visual", "type":"enumerated", "default":"general", "values": [ { "name":"general", //... } //... ] }, + { + "name":"visual", + "type":"include", + "default":"", + "values": [ + { + "name":"high-contrast", + "values": ["base/color/light/light.json5"] + } + ] + } ] } ```

Orthogonality

Within the possible dimensions we identify a concept of orthogonality to determine if one dimensions is independent of another. In laymen's terms, can a modifier change freely without needing to change how resolution of another modifier works. The following diagrams illustrate this point.

In this first diagram, two independent sets are used to resolve the final value. Any change to the Mode choice does not affect the theme set used.

flowchart TD
    Mode(Mode) --> Light
    Mode -->Dark
    Theme(Theme)-->Samurai
    Theme-->Knight
    Light-->Final[[Final]]
    Samurai-->Final

In this diagram, the choice of mode combined with the theme changes which set is used for final resolution. If the mode modifier is changed, the choice of potential candidates for theme changes as well

flowchart TD
    Mode(Mode) --> Light
    Mode -->Dark
    Light(Light)-->SamuraiL[Samurai:Light]
    Light-->KnightL[Knight:Light]
    Dark(Dark)-->SamuraiD[Samurai:Dark]
    Dark-->KnightD[Samurai:Dark]
    SamuraiL-->Final[[Final]]

In the above example we purposefully chose values such that the we had balanced options where each mode has the same options to choose from, lets now see an example where this becomes unbalanced

flowchart TD
    Mode(Mode) --> Light
    Mode -->Dark
    Light(Light)-->SamuraiL[Samurai:Light]
    Light-->KnightL[Knight:Light]
    Light-->PirateL[Pirate:Light]
    Dark(Dark)-->SamuraiD[Samurai:Dark]
    SamuraiL-->Final[[Final]]

In this case the choice for resolution becomes contextual. ONLY if the chosen mode is Light do you have the option of choosing the Pirate theme and if you do chose both Dark and Pirate you have made an invalid selection.

This is a basic example using only 2 modifiers but once other modifiers such as "Brand", "Surface", etc are included into the mix, more logic would be necessary to both handle the resolution of this, and also to express what combinations are allowed. This adds complexity to the overall solution.

Future extensions

We have attempted to keep the resolver structure as basic as possible outside of some minor additions like the name and description that could be useful.

As part of extensions for the spec, we might want to have a resolver potentially refer to another resolver so that certain calculated sets might be blackboxed. This might take the form of using a different type in the modifiers object to specifiy a resolver as opposed to an enumerated or include type which might reference a resolver.

This is only really useful if we cannot create precomputed token sets, but might have its uses otherwise. An example of such might be to relay context to another resolver to simulate properties such as surface logic.

In the example shown, if the visual modifier could not be normalized to be completely orthogonal to the mode, via the use of a semantic layer which could handle the mapping, ie

graph LR
  A["In-Scope Visual Tokens"]--> B
  B["Semantic Layer"] --> C["Output"]

and the choice of sets is dynamic because of it, eg src/tokens/functional/color/light/overrides/light.protanopia-deuteranopia.json5 vs src/tokens/functional/color/dark/overrides/dark.protanopia-deuteranopia.json5, then the output would depend upon the mode modifier for the visual modifier. To prevent overloading the complexity of one resolver for the different dimensions, one resolver would just read the output of another resolver either directly or through a precomputed set as previously mentioned. The alternative would be to change the resolver specification to express the acceptable combinations through the use of a tree structure that could be pruned. Looking back at our example of a non orthogonal modifier structure :

flowchart TD
    Mode(Mode) --> Light
    Mode -->Dark
    Light(Light)-->SamuraiL[Samurai:Light]
    Light-->KnightL[Knight:Light]
    Light-->PirateL[Pirate:Light]
    Dark(Dark)-->SamuraiD[Samurai:Dark]
    SamuraiL-->Final[[Final]]

Assuming this is valid and enforceable, this also reduces the space to only 4 sets as opposed to the 3 x 2 =6 permutations for each mode with each theme. For design systems that want to use a resolver spec as a source of truth, they could use it to construct a graph showing the different possible variations of a component, making it explicit which combinations are valid and invalid for the different dimensions. This could likely be useful in cases where the choice of the brand dimensions heavily affects the possible values of the other modifiers as brands might offer completely different combinations. This should likely be addressed by the use of seperate resolvers if they are heterogenous in their modifier usage, but regardless this could be more efficient for export tools.

As @romainmenke mentioned there will likely need to be some form of bridge connecting the possible values of a token and then providing some form of metadata to it to then be used in the various platforms, likely some form of output with conditional rules. Seeing as this is likely going to be dynamic as it needs to be evaluated at runtime (eg screen size) its outside the scope of what the resolver on its own might do without some additional system that could be embedded into a front end to do this.

A modifier that is not acceptable for this form of resolving is an example such as component state. A component can exist in multiple different potential states, eg Focused, Hover and in Error state, and each of these states might have different levels of precedence, such as the red color of the background always being shown.

Lastly the Primer examples shows an interesting point of using glob based selection of the sets. This could be useful in solving the issue shown above with regards to the file paths of the visual afflictions being dependent on the mode. If we potentially allow string interpolation based on the modifer inputs that could simplify the structuring and not require refactoring of semantic layers to enable this approach

For us, having distinct files would be unmanageable. We define hundreds and hundreds of tokens, currently with 18 different modes applied. Getting the full picture of what a token's value can be would be difficult, and we find that once the token system is set up, we update tokens rather than updating modes. Having a single file where a token showcases all its values make that easier to manage.

I'd love to see both formats supported by tools, though I believe what tools like Style Dictionary currently do (multiple files with a value per token per file) is too inconvenient for long-term maintenance workflows.

@Sidnioulz could you describe your curation process a bit further?

Also what purpose does seeing a token's value across themes provide? From my perspective, token values curated collectively by theme reduces context switching across themes because we commonly select values for tokens related to other existing values in the same theme. Seeing the token value across themes out of context of the larger theme doesn't seem helpful but I'm interested to know how it could be?

@SorsOps Really interesting proposal, with a lot of flexibility I think.

We are actually in the midst of setting up a flexible white label design system, which should be able to support an arbitrary amount of themes/modifiers. In order to do this, we‘ve come up with a custom style-dictionary implementation that can take a config built around the idea of what is called sets in your proposal.

Our implementation is a work in progress and much simpler than the one above, but brings two key differences:

Instead of explicitly naming filenames, we work with glob patterns to be able to reduce the amount of LOC while keeping a human readable format.
We are also able to filter the output based on a glob pattern that matches the name of each token. We implemented this, since we ran into situations where a designer might need a specific token (i.e. font-size for an icon font), that an engineer might not use (icons implemented as svg with a …-dimension-height token). This reduces bundle size.

As far as I understand your example, it is possible to granularly combine tokens from various inputs, but it isn’t possible to further filter them. This would mean the input token structure still influences the outcome, since you‘d need to carefully group certain tokens in certain files. I might be wrong though, so please feel free to clarify if thats the case!

We are able to modify each token names to get rid of arbitrary token group names such as default, which Figma Tokens still requires since token names can’t share group names.

For reference, here is the (very simple) TS definition of the current config implementation:

type ConfigToken = {
  path: string // Used as a glob pattern
  ignore?: string | string[] // Used as (a) glob ignore pattern(s)
  selector?: string // CSS Selector
  filename?: { // Filename modifier, used to create output folder structures
    prefix?: string
    suffix?: string
    replace?: {
      from: string
      to: string
    }[]
  }
  tokens?: {
    ignore: string[] // Mulitmatch pattern to ignore certain tokens
  }
}

type ConfigTokenGroup = {
  selector?: string // Optional CSS Selector for all entries in the group
  files: ConfigToken[]
  filename?: {  // Optional filename modifier for all entries in the group
    prefix?: string
    suffix?: string
    replace?: {
      from: string
      to: string
    }[]
  }
}

type ConfigOptions = {
  basePath?: strin
  fileType?: string
  log?: boolean
}

type TokenTransformConfig = {
  options: ConfigOptions
  tokens: {
    include?: ConfigToken[] | ConfigTokenGroup
    source?: ConfigToken[] | ConfigTokenGroup[]
  }
}

Taking cues from style-dictionary, tokens within include are always imported as values that might be referenced, whereas everything in source is parsed and transformed separately.

I'm unsure how any of this would fit into the proposal, but I do think it might be generally useful.

Hey @robinscholz.

Instead of explicitly naming filenames, we work with glob patterns to be able to reduce the amount of LOC while keeping a human readable format.

I saw in Primer they use the same. I understand the use case, but it would make it non deterministic to not know which files are necessary for the resolution, also assuming that the glob returns values in the same order which might cause a problem.

We are also able to filter the output based on a glob pattern that matches the name of each token. We implemented this, since we ran into situations where a designer might need a specific token (i.e. font-size for an icon font), that an engineer might not use (icons implemented as svg with a …-dimension-height token). This reduces bundle size.

I know what you mean. Trying to reduce the emitted tokens is ideal. This could be done through a post resolution step to choose the appropriate tokens prior to placing it in the style dictionary. I think this would be considered an edge case as we would either need to encapsulate the output and make it a single set output or potentially break it down into further resolvers for the pieces?

I think the takeaway is that we would not want to tie the support specifically to style dictionary as we would need to keep separation of concerns. Style dictionary has a very specific role handling transformation for platforms and rules for the output . I think this should be kept seperate from just the resolution step so we can keep simple tools for those stages

Gah! Sorry y'all I did not mean to close the issue just now. That was just my fat fingers on my phone!

Thanks so much for sharing the resolver proposal, @SorsOps. It's super interesting. I also had a brief play with the online tool you shared and it looks pretty sweet.

As you say, this moves the problem of how to handle tokens with variable values (for brands, themes, color schemes, information densities, etc. etc.) out of the scope of the DTCG format, which could help keep the spec simple for now. (aside: I do think there are some concepts within that resolver spec that might be nice to absorb into the spec in the future though)

However, it does raise at least one requirement for the DTCG format:

A single DTCG file must be allowed to contain references that cannot be resolved within that file

Otherwise, the files used as modifiers would not be able to contain aliases to tokens provided by the sets.

Currently the DTCG spec does not forbid this, but it doesn't explicitly allow it either. If we were to agree that we wanted to allow this, then I think we should explicitly state that in the spec to avoid differing interpretations (e.g. one tool assuming references must be resolvable within the same file and therefore rejecting files where that's not true).

(Btw, I think this current ambiguity in the spec was highlighted before in another issue, but I can't find it right now)

Thing is, do we want to allow that?

Personally, I quite like the idea that every DTCG file is self-contained. I think that can make them easier for humans to reason about. It could also make them more portable - e.g. the use-case mentioned above about referencing tokens from another DS's tokens. If you can cherry pick any .tokens.json file safe in the knowledge that it's not going to contain references that might not resolve, then I think that becomes easier.

What this thread is making clear is that there is a strong demand for being able to organise tokens into separate files. Forgetting about modes/themes/whatever for moment, that's desirable even if it's just to divide up a large collection of design tokens into more readable and manageable chunks.

I've got some ideas for how we could add that facility within the DTCG spec. However, I'll write it up as a separate issue and link to it here once I have.

Btw, I think this current ambiguity in the spec was highlighted before in another issue, but I can't find it right now

this one?

https://github.com/design-tokens/community-group/issues/123#issue-1190869199

Yup, that's the one. Thanks for fishing it out, @romainmenke!

re: composite tokens and multiple themes (which i believe token studio supports currently)

it doesn't seem like the current shape of the spec supports being in multiple "modes" at the same time

mainly thinking in terms of composite tokens each mode overrides a specific property? Maybe it's not a valid use case though

Sticking to the first post/request and the original issue, we should ignore tools/figma/plugins/third-party implementations and requests since the spec must be agnostic and must work for everyone.

Now, as the author of the original issue, in my opinion, theming and aliasing are the same thing. What I mean, once you can create an alias, you can create as many themes as you want, the only difference is the token name. There is no need to add a "specific way" to do theming... and "modes" is really a misleading name and a concept that IMHO should not be bound to the token itself. Design token should be simple, and hold a single raw value.

I'm posting here this really explanatory screenshot:

Following this important screenshot and the theming concept, we can achieve this with the current spec just with aliasing.

colors.tokens.json

{
  "$name": "Color Tokens",
  "red": {
    "$type": "color",
    "$value": "#ff0000"
  },
  "blue": {
    "$type": "color",
    "$value": "#0000ff"
  }
}

light-theme.json

{
  "$name": "Light Theme",
  "global": {
    "background": {
      "$type": "color",
      "$value": "{colors.red}"
    },
    "foreground": {
      "$type": "color",
      "$value": "{colors.blue}"
    }
  }
}

light-theme.json is just a set of semantic-named tokens that is transformed by the author for required platforms (transformation is a required step but outside the spec). For the web, for example, it could just become:

light-theme.css

:root {
 --global-background: #ff0000;
 --global-foreground: #0000ff; 
}

The only issue here is interoperability and alias resolving, no modes, no "theming" feature, or whatever. But since tokens must be always compiled/transformed, cross-file resolving should not be a big issue (but that's not my field)

The fact of having "modes" bound to the token is only a technical requirement for the Figma team and should not be the focus of the spec since it also violated one of the DT principles:

Design tokens don't change across implementations. They are constants.

Supposing we really want to bind everything to the token and single file, authors will end up with infinite JSON if they have to put everything in a single file. Also, tokens transformation could be a pain.

Warning Personal opinion ahead

Expand personal opinion

Personally, I don't like how this issue is now focused to satisfied only Figma technical requirements, instead of focusing on the spec, simplicity, and platform interoperability. If you really want to shape the spec around specific companies' requirements and put theming/transformation inside the spec, I'll probably never follow the spec and stick to Style Dictionary whereas as an author I can have control over token transformation over the platforms and where design tokens respect the core principles:

Design tokens don't change across implementations. They are constants.
Design tokens are platform-agnostic raw data; you must transform them into specific formats used by destination platforms, like the web, iOS, and Android.
Design tokens are non-semantic. They hold an absolute value.
Design tokens are the single source of truth to keep consistency across the system and platforms

(btw, principles we would violate if we add modes since the value of the token will change across the systems/platforms.

design-tokens / community-group

Native modes and theming support #210

Overview

Defining Modes

Fallbacks

Token Values

Value Resolution

Using new mode() option

Using postcss config

Features / User Preferences

Behavior

References as Solutions in Development

A Format to Structure Permutations

Does $value really need to be optional?

Could modes have additional metadata?

Should modes have $extensions?

Could modes have additional metadata?

Should modes have $extensions?

Should a tokens file be easily edited/maintained by a human in a text editor?

149 is still sitting open...and @romainmenke has posed some follow up questions we've yet to respond to there.

A. If yes - a tokens file should be easily edited/maintained by a human in a text editor

B. If no - tokens files will almost never be edited/maintained by a human in a text editor

Multi-file approach

Feedback

1. Definitions

2. Theming with Design Tokens (an Excerpt)

3. Formalizing a Spec Draft

1. Define the Capabilities of a Theme

2. Multiple Token Values

Validating the Spec

Open Questions

1. There are times where someone will care about and want to use multiple themes at the same time.

2. Theme metadata like features, tags, priorities, and theme types may be over-complicating the spec

Proposal: let's define themes with the least modifications to the current spec

Example: a dark theme and light theme with token aliases in a single file

Example: the same themes with multiple files

Example: creating a “patch”

Example: stacking themes

Example: defining themes on the token level

The downside to this approach

Ok, so how do translators know how to handle contexts?

1. There are times where someone will care about and want to use multiple themes at the same time.

2. Theme metadata like features, tags, priorities, and theme types may be over-complicating the spec

Example: a dark theme and light theme with token aliases in a single file

TL;DR

TL;DR

We will take all the feedback we receive here to evolve the spec according to requirements

Parameters

Background

Motivation

Why this approach over the current $themes approach?

Why is this preferred over embedding $modes?

Pros

Resolution logic

Example calculation

Usage with export tools

Resolution aliasing

Real world use case

Orthogonality

Future extensions

Does `$value` really need to be optional?

Should modes have `$extensions`?

Should modes have `$extensions`?

Why this approach over the current `$themes` approach?

Why is this preferred over embedding `$modes`?