Feature Request: Automate Changelogging

hoorayimhelping commented 4 years ago

Background

Currently, our process for change logging is to create a PR, consider if the change belongs in changelog, add an entry, then force push the whole thing, referencing the PR.

Until late September, we didn't have a changelog in ui at all. It's an important concept for product and product marketing, but it's something we've let slide as engineers.

Considerations

For a changelog to reference the PR it's part of, the PR has to be created before it can be linked in the changelog.

The relevant entries to include are feat and fix.

Should this be fully automated, or a script someone has to run?

danxmoran commented 3 years ago

The Telegraf team has written a script to generate a changelog at release time based on git history. It relies on squash-merge commit format to find PR numbers. We've decided to lock down our merge strategies as a step towards implementing a similar setup.

codyshepherd commented 3 years ago

evidently the flux team "has automatic change log based on commit history ... the only adjustment engineers hhave to make is to rebase/squash commits"

codyshepherd commented 3 years ago

Here's my writeup after digging into this.

Requirements

The Changelog is meant to be a human-readable summary of features and fixes in the corresponding software.

Release Branches

remove need for post-PR push to update Changelog for a given set of changes
remove need for manual check/update/consolidation of Changelog contents at release time
Changelog should summarize major changes (features and fixes) for each release, and provide an up-to-date summary of unreleased changes currently in the branch

Master Branches

should summarize major changes (features and fixes) to development (master or master-1.x) branches
need not track release branches, as this is redundant

Scheme Proposal

Release Branches

CHANGELOG.md updated by CircleCI during release process
between releases, unreleased changes tracked in a temporary file that is not commited to VCS, but is pushed to s3 (dl.influxdata.com)
- the unreleased changelog will be built nightly and will be stored/delivered alongside nightly builds

Master branches

CHANGELOG.md will be updated by CircleCI during nightly builds and committed to VCS at that time, in addition to being delivered to s3 alongside nightly build artifacts

Any Branch

Changelog updates will be handled by circleCI via use of the git cliff tool
Changelog tooling will also have limited support for git trailers, specifically the tag Changelog: false; any commit with that trailer will be left out of the Changelog
CircleCI automation will check during PRs for a small set of common updates to leave out of the Changelog; the canonical example of this is Flux updates -- if the flux dependency hash changes in the go.mod, the CircleCI check will enforce that the git trailer Changelog: false exists in the commit message.

danxmoran commented 3 years ago

if the flux dependency hash changes in the go.mod, the CircleCI check will enforce that the git trailer Changelog: false exists in the commit message.

We'll likely need to tighten this up... sometimes we / the query team will open a PR to fix a flux-related bug or add a flux-related feature, and doing so requires bumping the flux version in go.mod.

If we could detect PRs that only change the flux hash and only touch go.mod+go.sum, I think that would be a safe combo to enforce Changelog: false.

danxmoran commented 3 years ago

CHANGELOG.md will be updated by CircleCI during nightly builds and committed to VCS at that time

Does it need to be commited to VCS if it's also uploaded as a nightly artifact?

danxmoran commented 3 years ago

the unreleased changelog will be built nightly and will be stored/delivered alongside nightly builds

Calling out assumed sub-task: need to set up nightly builds for release branches

dgnorton commented 3 years ago

Changelog tooling will also have limited support for git trailers, specifically the tag Changelog: false; any commit with that trailer will be left out of the Changelog

I assume adding or removing a trailer afterwards is considered re-writing history (i.e., the SHA1 changes)? We'll need a way to update this after the fact I think. E.g., I merge a PR that shouldn't be in the changelog but forget to add the trailer. Then some weeks later we realize this and need to update it. If adding a trailer requires re-writing git history, we won't be able to do that.

codyshepherd commented 3 years ago

CHANGELOG.md will be updated by CircleCI during nightly builds and committed to VCS at that time

Does it need to be commited to VCS if it's also uploaded as a nightly artifact?

I would say yes because we don't have a great User Interface for the s3 bucket currently. If this is primarily for public consumption by non-power-users, just having it in s3 will likely generate constant questions from the community/support asking "where can I find the changelog." Better IMO to just also commit it to VCS.

That's just my two cents.

codyshepherd commented 3 years ago

Changelog tooling will also have limited support for git trailers, specifically the tag Changelog: false; any commit with that trailer will be left out of the Changelog

I assume adding or removing a trailer afterwards is considered re-writing history (i.e., the SHA1 changes)? We'll need a way to update this after the fact I think. E.g., I merge a PR that shouldn't be in the changelog but forget to add the trailer. Then some weeks later we realize this and need to update it. If adding a trailer requires re-writing git history, we won't be able to do that.

Git trailers are part of the commit message, so yes, updating the trailer after the fact would be tantamount to amending the commit history.

The scheme I am working on should still allow manual updates to the changelog for surgical edits like this. This is a good requirement to capture for test cases.

influxdata / influxdb