feat(build-cli): New command generate:releaseNotes

[tylerbutler]

I added a new command, flub generate:releaseNotes, to generate our release notes from input changesets. This command serves as a replacement for the generate:upcoming command. I used this to semi-manually generate the 2.1 release notes and explore what additional metadata is needed to produce the output we want.

In this implementation, we can add a second front-matter section to each changeset with the additional metadata we want. For example:

---
"@fluid-experimental/property-query": minor
---
---
section: fix
includeInReleaseNotes: false
---

The section ultimately represents a top-level section in the release notes. The sections and their headings are configurable in the root fluidBuild config. I added a release notes section for the SharedTree DDS because it is undergoing enough changes that a whole section is warranted, but over time the sections can change as our needs change.

Within each section, changes are sorted based on the date the changeset was created. This should mean that the oldest changes are listed first, but when changesets are added late then that won't be true.

Changes can be sorted before others in a section by setting the highlight field to true. Within a section, highlighted changes will be listed first.

The includeInReleaseNotes field can be used to omit a change from the release notes. This can be used in cases where a change should be noted in the individual package changelogs, but isn't really relevant at the release note level. A major dependency update is an example of such a change.

About the metadata approach

I'm not sure this is the best approach for metadata, but one advantage is that the existing changeset tools don't completely choke on the input with errors like:

Error: "blood-moon-rising" changeset mentions a release for a package "includeInReleaseNotes" but such a package could not be found.

That said, we'd still need to do some processing ourselves to prevent the second front-matter section from making it into the changelogs. So bottom line, there is more design work to do, and perhaps there will be little value in maintaining compatibility with the existing tools if our requirements aren't themselves compatible with those tools. As it is, I was able to use this to generate the notes and maintain them by manually adding the additional metadata, which will then be removed before merge so the downstream changelog tools work as expected.

If we imagine a small group of folks managing the release note content, we may be able to punt on adding the metadata at changeset creation time. Regardless, the person generating the notes can always make metadata adjustments at the last minute to affect the notes output.

Primary packages

Each change has a "primary package" which is the first package listed in the front matter, but that isn't really used in the current implementation. I'm leaning towards generic "sections" as being more useful than package groupings. Especially if the sections can be defined per-release so we can create a package-level section if we want, like I did for tree here.

The implementation is deliberately simplistic and should be considered an MVP. The markdown output is generated with a string builder rather than an AST.

Guidelines for reviewers

When reviewing, I suggest looking at these draft 2.2 release notes, which were created using this tool.

AB#9063

[msfluid-bot]

⯅ @fluid-example/bundle-size-tests: +245 Bytes

Metric Name	Baseline Size	Compare Size	Size Diff
aqueduct.js	457.41 KB	457.44 KB	⯅ +35 Bytes
azureClient.js	554.57 KB	554.62 KB	⯅ +49 Bytes
connectionState.js	680 Bytes	680 Bytes	■ No change
containerRuntime.js	258.08 KB	258.09 KB	⯅ +14 Bytes
fluidFramework.js	408.3 KB	408.31 KB	⯅ +14 Bytes
loader.js	134.04 KB	134.05 KB	⯅ +14 Bytes
map.js	42.13 KB	42.14 KB	⯅ +7 Bytes
matrix.js	146.42 KB	146.42 KB	⯅ +7 Bytes
odspClient.js	522.72 KB	522.77 KB	⯅ +49 Bytes
odspDriver.js	97.55 KB	97.57 KB	⯅ +21 Bytes
odspPrefetchSnapshot.js	42.61 KB	42.62 KB	⯅ +14 Bytes
sharedString.js	163.14 KB	163.15 KB	⯅ +7 Bytes
sharedTree.js	398.81 KB	398.82 KB	⯅ +7 Bytes
Total Size	3.3 MB	3.3 MB	⯅ +245 Bytes

---

Baseline commit: dc9a847bf65b9a0967a752934a468480ee50e04a

Generated by :no_entry_sign: dangerJS against c027192dc9d388a5f4abba85de43893993006d2d

[azure-boards[bot]]

✅ Successfully linked to Azure Boards work item(s):

• Task 9063

[Josmithr]

Within each section, changes are sorted based on the date the changeset was created. This should mean that the oldest changes are listed first, but when changesets are added late then that won't be true.

Could we add a custom metadata field to annotate the commit the changeset is intended to be associated with? We could get the correct sorting that way. Probably outside the scope of this PR, but maybe worth considering longer term?

Should the additional metadata include a manual "order" or "weight" field that is used to sort within the sections? Seems a bit too fiddly for me but with features in particular, we might want to highlight some first.

Would also address this. I personally don't think devs should be explicitly messing with ordering and such.

[alexvy86]

What do you think of the metatadata? What's missing? Should the primary package be explicit rather than inferred from package order?

Slight concern that people have to know the possible sections when writing a changeset. If using pnpm changeset the tool might help, but what'll happen if someone writes/copy-pastes-edits them by hand and specifies a section that doesn't exist?

Should the additional metadata include a manual "order" or "weight" field that is used to sort within the sections? Seems a bit too fiddly for me but with features in particular, we might want to highlight some first.

I agree it seems too fiddly. Reordering of the changes in the notes I think is more of an editorial decision before we publish the notes, so I wouldn't try to bake it into the tools. If anything, maybe adding a metadata field for "I want special attention when release notes are generated"? Whether for ordering, more details, things that couldn't be added when the changeset was written, "maybe I don't want to end up in the release notes", etc. The person with the holistic view of all changes is probably in a better position to look at that request and take the appropriate action, and can reach out to the changeset author if necessary.

What about the list of packages affected at the bottom of each section? Useful?

Not sure about this... does it mean every single change (or close to) will have fluid-framework listed there?

Should each change include more info? E.g. link to the PR, the commit, etc?

That would be lovely. I like being able to go look at the actual change if I find something particularly relevant or interesting in release notes of other projects. I wouldn't make it a top priority though.

Other comments

I think a TOC at the top of the release notes would be nice, just listing the sections and the summaries, with links to the appropriate header. Especially when the notes get large, that would help people that need to do quick scans. Right now I think it's relatively easy to miss a header in between all the surrounding text.

[Josmithr]

What about the list of packages affected at the bottom of each section? Useful?

I think it's at least potentially useful. I would vote to keep it.

Should each change include more info? E.g. link to the PR, the commit, etc?

PR + Commit - yes 🙂

Changes that affect more than one package end up in both packages' changelogs. This is OK for some changes, but for packages like fluid-build it can be confusing if the change itself doesn't describe what it affects. For example the 2.1 changelog entry for fluid-framework is almost exclusively about tree, but it's not clear from the changset content - either the title or the body - that a change affects the SharedTree parts of fluid-framework. Should we care?

My instinct says that the best solution to this problem is to consider all of the affected packages when writing the changeset description. If it affects more than just tree, the notes should probably call out the external impacts.

An alternative solution for things like trivial dep updates and such in a subset of packages would be to omit those packages from the changeset metadata. It might even make sense to filter out dev-dependency-only changes when determining "affected packages".

[tylerbutler]

Could we add a custom metadata field to annotate the commit the changeset is intended to be associated with? We could get the correct sorting that way. Probably outside the scope of this PR, but maybe worth considering longer term?

Yes; the drawback is we'll need to write more custom code to create changelogs. Right now we use the changeset tools for that piece. We have a very thin wrapper around it. Not a huge deal just another thing to do.

But keep in mind that the changeset is already automatically associated with the commit that added it. It's only when we add changesets late that we want to manually set a commit.

Should the additional metadata include a manual "order" or "weight" field that is used to sort within the sections? Seems a bit too fiddly for me but with features in particular, we might want to highlight some first.

Would also address this. I personally don't think devs should be explicitly messing with ordering and such.

I don't think devs would set this - it's more for the person generating the release notes, to give them control without manually changing the output (that's proven to be a major pain).

Slight concern that people have to know the possible sections when writing a changeset. If using pnpm changeset the tool might help, but what'll happen if someone writes/copy-pastes-edits them by hand and specifies a section that doesn't exist?

I'm not too worried about the devs getting this metadata 100% correct. I think there will always need to be an editorial step, and metadata can be adjusted at that point.

If anything, maybe adding a metadata field for "I want special attention when release notes are generated"? Whether for ordering, more details, things that couldn't be added when the changeset was written, "maybe I don't want to end up in the release notes", etc. The person with the holistic view of all changes is probably in a better position to look at that request and take the appropriate action, and can reach out to the changeset author if necessary.

I like this idea, but I think we can add it later. Like an "important" flag or something.

Not sure about this... does it mean every single change (or close to) will have fluid-framework listed there?

No, only packages that are also exported through fluid-framework, which isn't many. Mostly DDSes; the lower-layer APIs are consolidated in fluid-static and are used in fluid-framework but not re-exported.

I think a TOC at the top of the release notes would be nice, just listing the sections and the summaries, with links to the appropriate header. Especially when the notes get large, that would help people that need to do quick scans. Right now I think it's relatively easy to miss a header in between all the surrounding text.

Good idea. I'll add that.

It might even make sense to filter out dev-dependency-only changes when determining "affected packages".

We do filter out private packages and some scopes I think, but that's only at changeset generation time.

[kashms]

• I like the configurable sections but probably something we'll be doing infrequently. Tree is a great addition

Within each section, changes are sorted based on the date the changeset was created. This should mean that the oldest changes are listed first, but when changesets are added late then that won't be true. Regardless, the sorting at that level is meant to ensure the output is stable, not for organization. The dates are not included in the output.

• Ideally we want this sorted based on impact of the change but I'm not sure how we do ordering at change time. I like Alejandro's suggestion of having a boolean value for highlightChange which automatically moves the changeset towards the top of the section. That is, all highlightChange items get bubbled up; secondary sort can still be timing

Each change has a "primary package" which is the first package listed in the front matter, but that isn't really used in the current implementation. I'm leaning towards generic "sections" as being more useful than package groupings. Especially if the sections can be defined per-release so we can create a package-level section if we want, like I did for tree here.

• I agree, the package grouping is something we should move away from. Consumers of the Public API would prefer API groupings (which don't always map to packages). Sections like "Tree changes", "Audience changes" in addition to the catch-all types like "new features", "deprecations" and "bug fixes" make sense to me.

• What do you think of the metatadata? What's missing? Should the primary package be explicit rather than inferred from package order?

• I think implicit is fine, I think the changeset text should be descriptive enough to help the readers know the API in reference

• Should the additional metadata include a manual "order" or "weight" field that is used to sort within the sections? Seems a bit too fiddly for me but with features in particular, we might want to highlight some first.

• See comment above about "highlightChange" metadata field

• What about the list of packages affected at the bottom of each section? Useful?

• Yea, I'd say let's leave it there for now. It does get repetitive in sections that are largely aligned with a package aka tree but are critical in generic sections like "bug fixes". Maybe have a flag in fluidBuild config for sections to disable the package affected? This way the tree section can exclude the packages affected section from all its changeset? just a quick thought.

• Should each change include more info? E.g. link to the PR, the commit, etc?

+1 for link to PR, I know React also includes a link to the author but that might be OP. If there's an easy way to link to the relevant API docs without adding to the changeset writer's burden, I'd also favor that. For example, if the changeset talks about SharedString, I'd appreciate if it linked to the SharedString docs.

• Changes that affect more than one package end up in both packages' changelogs. This is OK for some changes, but for packages like fluid-build it can be confusing if the change itself doesn't describe what it affects. For example the 2.1 changelog entry for fluid-framework is almost exclusively about tree, but it's not clear from the changset content - either the title or the body - that a change affects the SharedTree parts of fluid-framework. Should we care?
• I'm not sure I follow this, but this is about changeset files and not release notes, right? If so, I wouldn't worry about this :)

[changeset-bot[bot]]

⚠️ No Changeset found

Latest commit: c027192dc9d388a5f4abba85de43893993006d2d

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[tylerbutler]

/azp run Build - client packages

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[tylerbutler]

/azp run server-routerlicious

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).