[rush] Make ~/.rush-user/settings.json interoperable between Rush versions

[octogonz]

Summary

Rush strictly validates the schema for its JSON config files. This provides a great user experience for catching mistakes such as a misspelled setting or unrecognized feature. Interoperability between Rush releases is not a concern because the version selector ensures that config files in a given branch will always be parsed by a specified Rush release. (Whenever a Rush command is invoked, the version selector reads rushVersion from rush.json and automatically installs that version of the tool, ensuring a completely deterministic build.)

The build cache feature recently introduced an experimental new config file ~/.rush-user/settings.json. Because it is located in the user's home directory, it must be readable by different Rush versions (for example when building old branches that install an old version of Rush). So unlike other the config files, this one needs to be interoperable.

The problem: If a future version of Rush adds a new field to this file, all previous versions of Rush will report a validation error when they try load it.

How can we fix this? Some possible ideas...

Option 1: Loosey-goosey

Instead of having a strictly enforced schema, we could model settings.json like package.json, which is basically a bag of arbitrary key/value pairs with no validation. If a setting is misspelled, or not supported, or added under the wrong section, then it will be silently ignored. Once a field has been introduced, its meaning/syntax can never be changed without breaking some previous release.

Option 2: Versioned config files

Add a suffix to the file such as settings-5.47.json, settings-5.49.json, settings-6.0.json etc. The config file suffix corresponds to the Rush release when that setting was introduced, but skipping over Rush versions that did not affect the schema.

Maybe Rush could help users to manage this. For example, if it finds any settings-*.json files, then but a version is missing for the current Rush release, then it would print a warning about that.

Option 3: Config loader package

Suppose we created a small NPM package @rushstack/rush-config whose job is to load ~/.rush-user/settings.json and present the settings via a TypeScript API. The TypeScript API is carefully designed so that API consumers are guaranteed to work perfectly with every future release of the @rushstack/rush-config package. (A simple way is to never remove or rename an API item. A more advanced way is for the caller to request a specific API contract version, similar to DirectX feature levels.)

The idea is that when an old version of Rush wants to load ~/.rush-user/settings.json, it would do so using the latest release of @rushstack/rush-config. It could download it on demand using a mechanism like install-run-rush.js.

To avoid download-on-demand we could ask the user to install @rushstack/rush-config globally, or maybe under ~/.rush-user/node_modules/@rushstack/rush-config/.

[D4N14L]

Why not version the config files in the file themselves? Including a "version" property in the JSON would allow us to upgrade/warn about deprecated features without requiring user intervention (load, validate/warn, set current version, save), and setting/updating the version is as easy as saving the file. This is the method that PNPM follows for versioning their lockfile, and it's worked quite well in terms of backwards/forwards compatibility. We could version the file with Rush or otherwise, though the latter is more ideal, since it would make it easier to validate specific schemas against specific versions (since Rush versions so often)

[octogonz]

Currently the file schema looks like this (with "version": 1 added to it):

~/.rush-user/settings.json

{
  "version": 1,

  /**
   * Optionally override the default location of the cached tarballs:
   */
  "buildCacheFolder": "/temp/my-custom-folder/"
}

Now suppose hypothetically we want to revise the schema to be more granular, like this:

{
  "version": 2,

  "buildCache": {
    /**
     * Optionally override the default location of the cached tarballs:
     */
    "tarballsFolder": "/temp/my-custom-folder/",

    /**
     * Optionally override the default location where state files get stored:
     */
    "stateFilesFolder": "/temp/my-custom-folder2/"
  }
}

Now suppose that I pull an old Git branch from 3 months ago and try to build it. What is the old Rush supposed to do when it tries to load "version": 2?

[D4N14L]

My guess would be that it should attempt to load and validate. If there's no clear upgrade path from one property to another, or support for a property was dropped, we should warn the dev that the property will be ignored or removed/commented out from the settings (since commenting allows the dev to go back and modify their existing property manually). The versioning is just an attempt to provide an upgrade path... I don't think it means we should always have one. Breaking changes happen. But in those cases, the dev should at least know that it's happened, and be given an opportunity to resolve manually (and preferably with a link to the docs).

[D4N14L]

Ah, sorry, I see what you're saying... maybe we just have to eat the failure there and provide guidance to delete the file? The filename versioning would have the same issue since the settings file wouldn't be able to be found (it would be postfixed with the version). If we really really wanted to version while allowing full back compat, we could do something kinda-gross-but-still-reasonable like versioning in a comment within the JSON, so then we can have a version without needing it to be a part of the schema.

[octogonz]

Well, with Option 2 those two examples would go in ~/.rush-user/settings-1.json and ~/.rush-user/settings-2.json. You would need to maintain the same settings in two places, but it wouldn't be too confusing, since Rush would tell you which one it is using.

Whereas with Option 3 you would specify buildCache.tarballsFolder and the old Rush release would magically interpret that setting as its buildCacheFolder value. Also the old Rush release could let you know that it is ignoring buildCache.stateFilesFolder because it doesn't support that setting, while still validating that the setting has correct syntax.

[octogonz]

Option 4: Put the user settings under the monorepo folder

Instead of ~/.rush-user/settings.json, require users to specify their settings as my-repo/common/config/user/settings.json which is not tracked by Git. It is the user's responsibility to copy+paste this file between their repo enlistment folders, and fix up any incompatible settings.

[dmichon-msft]

I'd favor specifying a version in your config; if the specified config version is newer than the expected version, rush prints a warning that it is incompatible and ignores it, would be my vote. Specify the oldest version in your config that is compatible with the settings used.