improvement(build-tools): Add build-infrastructure package

This PR introduces a new package containing types and helper functions that will be used across multiple build-tools packages, including @fluidframework/build-tools and @fluid-tools/build-cli.

The primary purpose of this package is to provide a common way to enumerate Packages, Release Groups, and Workspaces across a Fluid repo.

The package is private and nothing depends on it yet, so it should cause no problems when merged in its current state.

Background

Today both fluid-build and build-cli (flub) need to understand the layout of the repo - what packages belong to what release groups, what folders are workspace roots vs. independent packages, etc. We have been using the types and classes from fluid-build in the flub code, but the relationship has always been strange because flub has a concept of release groups while fluid-build uses a MonoRepo class which is analogous to a Workspace.

Conceptually, Release Groups and Workspaces are distinct, but in code there is not a strong representation of the "release group" concept. In flub we use the MonoRepo class and a type ReleaseGroup = string together as a stand-in for the ReleaseGroup concept which has meant that release groups and workspaces (the MonoRepo class) are more or less the same thing in practice.

That has always been a very weird situation that has gotten complicated as the repo has grown, and now we find that we want the benefits of interdependencies in between packages provided by workspaces but the flexibility of releasing and versioning groups of packages within those workspaces. These changes are a step towards that future.

The goal

The eventual goal state is to have all the shared types and data about the repo layout and other shared code in the build-infrastructure package, and then have flub and fluid-build be in individual packages. They shouldn't depend on each other - all the shared logic - especially the types - should be in the infra package.

To do before merge

[ ] There is no way in the current PR to load the repo from the old repoPackages config. I think it is feasible to write a load function that works with the old config, which will make it easier to transition to the new config format.

See the package README in this PR for more details. Excerpts are included here for convenience.

API Overview

The API is built around four key types which form a hierarchy: IFluidRepo, IWorkspace, IReleaseGroup, and IPackage. For the purposes of this documentation, the terms "Fluid repo," "workspace," "release group," and "package" generally refer to these types. There are also load functions provided to create instances of the types. Concrete implementations are not exported.

What about "independent packages?"

In the v0 version of build-tools, we have the concept of "independent packages:" packages that are not part of a release group and are released independently. This concept no longer exists. There are only release groups. Packages that release independently can either be part of a single-package workspace (and release group), or they can be part of another larger workspace, contained within a single-package release group.

Configuration

Configuration for the repo layout is stored in a config file at the root of the repo. This can either be part of the fluidBuild.config.cjs file in the repoLayout property, or in an independent config file named repoLayout.config.cjs (or js).

Example

The following example configures three workspaces demonstrating the three archetypes - a workspace with multiple release groups, a workspace with a single release group that contains multiple packages, and a workspace with a single release group that contains a single package.

repoLayout: {
  workspaces: {
    // This is the name of the workspace which is how it's referenced in the API.
    "client": {
      // This workspace is rooted at the root of the Git repo.
      directory: ".",
      releaseGroups: {
        // This key is the name of the release group. It must be unique across all
        // release groups.
        client: {
          // The include property can contain package names OR package scopes. If 
          // a scope is provided, all packages with that scope will be a part of
          // the release group.
          include: [
            // Include all the Fluid Framework scopes and packages except for
            // @fluid-example.
            "@fluidframework",
            "@fluid-experimental",
            "@fluid-internal",
            "@fluid-private",
            "@fluid-tools",
            "@types/jest-environment-puppeteer"
            "fluid-framework",
          ],
          // A release group can have an OPTIONAL root package. This package
          // is typically private and is similar to the root package for a workspace.
          // This release group root package may be useful to store scripts or other
          // configuration only applies on the release group, 
          rootPackageName: "client-release-group-root",
          defaultInterdependencyRange: "workspace:~",

          // A release group may have an ADO pipeline URL associated with it. This
          // URL is used to provide direct links to the pipeline when running releases.
          adoPipelineUrl:
            "https://dev.azure.com/fluidframework/internal/_build?definitionId=12",
        },
        examples: {
          // This release group contains only the @fluid-example packages.
          include: ["@fluid-example"],
          rootPackageName: "examples-release-group-root",
          defaultInterdependencyRange: "workspace:~",
        },
        // If any packages in the workspace don't match a release group, loading the
        // repo layout config will throw an error.
      },
    },
    "build-tools": {
      // This workspace is rooted in the build-tools folder. This folder must contain
      // a workspace config. The specific config depends on the package manager being used.
      directory: "./build-tools",
      releaseGroups: {
        // Release groups can have the same name as workspaces, but all release group names
        // must be unique regardless of the workspace they belong to.
        "build-tools": {
          include: [
            // Include all Fluid Framework scopes. Only packages contained in the workspace
            // will be included, so it is safe to use the same scopes in multiple release
            // group definitions as long as they're in different workspaces.
            "@fluidframework",
            "@fluid-example",
            "@fluid-experimental",
            "@fluid-internal",
            "@fluid-private",
            "@fluid-tools",
          ],
          rootPackageName: "build-tools-release-group-root",
          defaultInterdependencyRange: "workspace:~",
          adoPipelineUrl:
            "https://dev.azure.com/fluidframework/internal/_build?definitionId=14",
        },
      },
    },
  },
}

Loading a Fluid repo from a configuration file

To load a Fluid repo, you use the loadFluidRepo function. You can pass in a path to a Git repository root, or if one is not provided, then the Git repository nearest to the working directory can be used.

This function will look for a repo layout configuration in that folder and load the workspaces, release groups, and packages accordingly and return an IFluidRepo object that includes Maps of workspaces, release groups, and packages as properties.

Metric Name	Baseline Size	Compare Size	Size Diff
aqueduct.js	461.15 KB	461.18 KB	⯅ +35 Bytes
azureClient.js	559.19 KB	559.24 KB	⯅ +49 Bytes
connectionState.js	680 Bytes	680 Bytes	■ No change
containerRuntime.js	261.99 KB	262 KB	⯅ +14 Bytes
fluidFramework.js	399.76 KB	399.77 KB	⯅ +14 Bytes
loader.js	134.26 KB	134.28 KB	⯅ +14 Bytes
map.js	42.39 KB	42.39 KB	⯅ +7 Bytes
matrix.js	146.56 KB	146.56 KB	⯅ +7 Bytes
odspClient.js	526.47 KB	526.52 KB	⯅ +49 Bytes
odspDriver.js	97.72 KB	97.74 KB	⯅ +21 Bytes
odspPrefetchSnapshot.js	42.78 KB	42.79 KB	⯅ +14 Bytes
sharedString.js	163.26 KB	163.26 KB	⯅ +7 Bytes
sharedTree.js	390.27 KB	390.28 KB	⯅ +7 Bytes
Total Size	3.3 MB	3.3 MB	⯅ +245 Bytes

microsoft / FluidFramework