Status object: simplified audit/decision history

[hereinthehive]

This is something I'd been thinking about before joining. A state/status object feels extensible and more descriptive. I'd also argue that the deleted state is valid as you might want to capture why a value was deleted.

With concepts like this, perhaps we're touching on whether tokens should have their own version of an ADR so there's an audit/history around choices/decisions made?

{
  "colors": {
    "action": {
      "$value": "#ff0000",
      "$type": "color",
      "$status": [{
        "$state": "deprecated",
        "$description": "We'll be replacing this soon, please check out how the replacement value works for you",
        "$value": "#00ff00",
        "$timestamp": "20220707111111",
        "$note": "We changed the color based on user research",
        "$toolstamp": "Figma",
        "$author": {
          "$name": "Dan Donald",
          "$email": "dan@zeroheight.com"
        }
      },
      {
        "$state": "created",
        "$timestamp": "20220128093348",
        "$author": {
          "$name": "Dan Donald",
          "$email": "dan@zeroheight.com"
        }
      }]
    }
  }
}

The rationale here is that as tokens when used to their full potential are a real business asset, it's useful to know when and why a decision was made. Seeing a simple history allows us to have context for a value but also allow any given tool to write to it and have that shared understanding in a way we haven't to date. Arguably the notion of 'toolstamp' is a naive way of suggesting there should be a reference to what tool was used as part of the audit.

Really curious to hear your thoughts!

I think having a $state property will be really beneficial to the spec as well.

In our implementation, we have a state property that can take one of the following states:

Experimental → Active → Deprecated → Sunset → Deleted

• Experimental: For introducing tokens for internal validation and trail purposes
• Active: This token is currently actively being used. Tooling will help to ensure tokens are used.
• Deprecated: This token has been flagged for future deletion and should no longer be used.
• Sunset: This token has been marked for upcoming deletion and strictly should not be used.
• Deleted (hard): This token has been fully removed from the tokens API

Our tooling leverages this along with a replacement property to perform automatic or controlled migrations via tooling such as eslint, stylelint, figma etc.

For example, a deprecated token with a replacement can be warned against and autofixed by eslint. A sunset token will produce an error and can be autofixed and so on.

{
  link: {
    pressed: {
      attributes: {
        group: 'color',
        state: 'deleted',
        replacement: 'color.link.pressed',
        description: 'Use for links in a pressed state',
      },
  },
}

We've had to build these tools ourselves, but the great thing about having an industry-wide spec like this one is that both community and proprietary tooling can leverage this concept to make token lifecycles a standard and consistent behaviour. Particularly helpful for the maintenance and evolution of tokens over time.

Originally posted by @hereinthehive in https://github.com/design-tokens/community-group/issues/118#issuecomment-1196516422

[TravisSpomer]

One issue I see is that this would only allow you to track changes made in a single tokens file. But let's say you have a global tokens file, and both a light theme file and a dark theme file each with the same tokens defined with different values. (See #2 for lots of examples of how people might do things like this.) Having change history be specific to each file makes it harder to track an update that moves a token from the global file into the theme files, which would touch all three. Tooling could obviously still update the change history for you in all three files, but it seems like it would be a lot easier to browse and understand changes in an external change tracking system like (ugh) Git.

[hereinthehive]

Fair comment!

I guess I see that for an individual token, you have many use cases. If it's an alias, it effectively defers to the originating token for the history but may have a status attached to it. Moving a token is interesting - perhaps that's actually a remapping? One deprecating and assigned to the new one so the audit trail is still there?

On the whole, I don't see many changes needed. If the core values, however they're expressed haven't changed, the audit log would be in the change of relationship in the component-level token or something. The chain is circumstantial with transitory relationships

[lauthieb]

@hereinthehive it's interesting your proposal.

Maybe that could be aside from the design tokens file. In order to avoid the mix between the current source of truth and its history/changelog?

Btw, as a workaround, I think with the current draft you can achieve this by create a specific file for your deprecated tokens like deprecated.json which is a tokens set that only contains your old tokens and use a $extensions custom field to explain why it is deprecated (by redirecting to a CHANGELOG.md file, a GitHub release note, a git tag, or text.

[hereinthehive]

@lauthieb I think part of the structure here is not just about deprecation but the full history that could be written across products/touch-points that interact with the tokens. Knowing when it was added, why and by who becomes more valuable over time when tokens truly become the core of an organisation's site/app/design strategy.

I agree that it certainly could get bulky! Perhaps a token.history.json file might work? The issue there becomes the fragility of maintaining relationships between nomenclature than might change over time. Use IDs as reference?

The argument against might be that Git does this well already, but as we know tokens might never reach Git - it depends entirely on a given set-up...so some kind of changelog/history sounds like the kind of maturity we may need in an interoperable space

Really curious what others think too!

[romainmenke]

I still think this is to be solved by git or something similar. GitHub is just a nice user experience on top of git.

The same can be done for design tokens.

We don't build version control into other file formats because generic solutions like git already exist.

[hereinthehive]

@romainmenke Of course, if there's a way to have design tools use Git under the hood for this, that would make perfect sense. As they can be hand authored and not necessarily going to touch version control (on a design-weighted team for example), this is just exploring a more portable implementation. More simplified, it may have a limited number of states...?

[jeromefarnum]

I wonder if this concern is something better tackled using $extensions records. While I certainly could see some teams desiring this level of tracking for their tokens, I would also guess many other teams wouldn’t need such granular details and can address their change management needs through other approaches (git being one example).

I could see a tool that registers and tracks $extensions records for this data. Teams could opt in to that tool/extension as they see fit.

[hereinthehive]

@jeromefarnum That's a fair comment. I'd still wonder if a few basic status captures might be helpful with more detailed data offset-up as you suggest. IMHO it's only as things mature that this kind of data is useful so storing created, last modified, deprecated, etc may be enough alongside the token to give it context with the option of more details for a full change log for extensions...but this would need to follow a pattern to work cross-platform, which then feels like something that should have a spec

[ilikescience]

+1 to letting this be solved by tools in $extensions or git or underlying database - no need for an additional status or state.

[drwpow]

+1 to letting this be solved by tools in $extensions or git or underlying database - no need for an additional status or state.

I agree with this—embedding history into the spec like this would balloon into quickly-unmanageable filesizes for larger design systems. Plus it would likely be limiting in many ways, since many concepts of versioning won’t ever map 1:1 with tokens (e.g. tokens will get deleted/merged/mutated in ways that will make a simple linear history difficult to trace). Plus, may design systems may want to define their own set of versioning principles, and come up with their own “changelogs” as they see fit, without restrictions.

I am in favor of the underlying $deprecated proposal because there is sufficient prior art there. But for versioning/auditing, this feels more deferring to Git or other existing protocols for this would be ideal.