search

prisma-labs / team

👩‍🔬👨‍🔬Past sprint demo recordings & more
1 stars 1 forks source link

dripip release tool #2

Open jasonkuhrt opened 5 years ago

jasonkuhrt commented 5 years ago

Supports https://github.com/prisma-labs/nexus-prisma/issues/379

image

This diagram is a visual representation of the release workflow explained below.

dripip

dripip is a CLI for releasing npm packages. It bakes in the concepts of stable-version, pull-request-version, and next-version releases. It assumes the project adhears to semantic versioning and conventional commits. It is designed for extreme simplicty and CI/CD.

Principals:

Release Flows

dripip supports three complementary release flows:

dripip stable

This bumps the package by a conventional commit calculation against the commits since last stable release.

    • [x] Assert HEAD is not a stable release.
    • [x] Calculate new version:
        • [x] Find the last stable version (git tag on the current branch). If none use 0.0.1.
        • [x] Calculate the semver bump type. Do this by analyizing the commits on the branch between HEAD and the last stable git tag. The highest change type found is used.
        • [x] Bump last stable version by bump type, thus producing the new version.
    • [x] Run publishing process:
        • [x] Change package.json version field to be new version.
        • [x] Run npm publish --tag latest.
        • [x] Undo package.json change.
        • [x] Run git tag {newVersion}.
        • [x] Run git tag next (normally should already be present)
        • [x] Run git push --tags.
        • [ ] Create a GH release titled by the version and pointing at the git tag version.

dripip preview

This publishes pre-releases like 1.2.3-next.1, 1.2.3-next.2, ... and so on that increment until finally resetting upon the next stable release.

    • [x] Assert HEAD has no release.
    • [x] Assert is not a PR
      • [x] or --no-pr
    • [x] Calculate new version:
        • [x] Find the last pre-release on the current branch. Take its build number. If none use 1.
        • [x] Calculate the semver bump type. Do this by analyizing the commits on the branch between HEAD and the last stable git tag. The highest change type found is used. If no previous stable git tag use 0.0.1.
        • [x] Bump last stable version by bump type, thus producing the next version.
        • [x] Construct new version {nextVer}-next.{buildNum}. Example: 1.2.3-next.1.
    • [x] Run publishing process:
        • [x] Change package.json version field to be new version.
        • [x] Run npm publish --tag next.
        • [x] Undo package.json change.
        • [x] Run git tag {newVer}.
        • [x] Run git tag next.
        • [x] Run git push --tags.
        • [x] Creates a GH pre-release called next at tag next If next release exists (it always will except for the first release ever) then do nothing (the force push updating next tag should be enough to make the GH next release pointing at a new SHA.

dripip preview on pull-requests

This publishes pre-releases like 0.0.0-pr.1.8c01648, 0.0.0-pr.2.8y2716f. "On pull-requests" is determined by dripip by looking for well known environment variables exported by CI providers. The CI providers currently supported are GitHub and CircleCI. This implies we are optimizing for running on CI systems. If you want to force this behaviour say from your machine add the --pr flag.

    • [x] Must be a PR and no --no-pr OR --pr present.
    • [x] Assert that not both --no-pr and --pr are present.
    • [x] Assert HEAD has no release.
    • [x] If --pr passed, use same algorithm used by hub show pr to infer pr number.
    • [x] Calculate new version:
        • [x] Construct new version {0.0.0}-pr{prNum}.{shortSHA}. Example: 0.0.0-pr.121.8y2716f.
    • [x] Run publishing process (no git tags here):
        • [x] Change package.json version field to be new version.
        • [x] Run npm publish --tag pr.{prNum}.
        • [x] Undo package.json change.

Version Branches

dripip supports version branches defined below. The three release flows receive special integrity checks ensuring sound coordination with version branches.

  1. A version branch is a branch whose name is a valid non-release-candidate semver value with optional v prefix.
    • [ ] On such a branch it is only possible to make releases that do not bump the part locked in the branch name. For example: Given v2 branch only patch and minor releases are possible; Given v3.3 branch only patch releases are possible.
    • [ ] On the parent of such a branch it is only possible to make releases of a higher order than the locked part of the version branch. For example: Given a v2 branch all subsequent parent branch releases must be v3.0.0 or higher; Given a v3.3 branch all subsequent parent branch releases must be v3.4.0 or higher.
    • [ ] It is important that releases on a version branch do not collide with releases anywhere else, such as on the parent branch. dripip version-branch <vb-ver> helps with this. It:
      • [ ] Creates a new git branch
      • [ ] tags the commit it branched off from with vb<vb-verb>

    The special tag vb<vb-verb> helps dripip do the right thing with respect to points 2 & 3 above.

    • [ ] If you opt to not use dripip CLI to create your version branch then manually create the tag as specified here.

    • [ ] If we find a reliable way in the future to correctly and effeciently rely on git CLI alone to enforce 2 & 3 we will, thus dropping the extra tag signal.

Release Notes

    • [ ] Release notes will be added to the GitHub release.
    • [ ] Release notes will be produced in Markdown format.
    • [ ] Release notes will be exportable as JSON.

    • [ ] Release notes for a single release will look like:

      <!-- @META { "checksum": "...", "dripipVersion": "..." } -->

<!-- @CUSTOM { "side": "start" } -->
<!-- @CUSTOM { "side": "end" } -->

#### BREAKING CHANGES

<!-- @CUSTOM { "side": "start" } -->
<!-- @CUSTOM { "side": "end" } -->

* [next.#] <short-sha>(link) <commit message> [ty <gh-user-name>(link)!]

  <commit message body>

* ...

<!-- @CUSTOM { "side": "start" } -->
<!-- @CUSTOM { "side": "end" } -->

#### Features

<!-- @CUSTOM { "side": "start" } -->
<!-- @CUSTOM { "side": "end" } -->

* [next.#] <short-sha>(link) <commit message> [ty <gh-user-name>(link)!]

  <commit message body>

* ...

<!-- @CUSTOM { "side": "start" } -->
<!-- @CUSTOM { "side": "end" } -->

#### Fixes

<!-- @CUSTOM { "side": "start" } -->
<!-- @CUSTOM { "side": "end" } -->

* [next.#] <short-sha>(link) <commit message> [ty <gh-user-name>(link)!]

  <commit message body>

* ...

<!-- @CUSTOM { "side": "start" } -->
<!-- @CUSTOM { "side": "end" } -->

#### Improvements

<!-- @CUSTOM { "side": "start" } -->
<!-- @CUSTOM { "side": "end" } -->

* [next.#] <commit message> <short-sha>(link) [ty <gh-user-name>(link)!]

  <commit message body>

* ...

<!-- @CUSTOM { "side": "start" } -->
<!-- @CUSTOM { "side": "end" } -->

<!-- @CUSTOM { "side": "start" } -->
<!-- @CUSTOM { "side": "end" } -->
      • [ ] The @META checksum is derived from the generated release notes contents. Later on dripip can check contents against the checksum to easily detect drift (manual edits).
      • [ ] The @META dripipVersion contains the same piece of data that would be in packge.json version snapshoted at the time of generation. This is useful to permit document format migrations from one version of dripip to the next.
      • [ ] The @CUSTOM directives permit users to customize their release notes while preserving the ability for dripip to update them later on.
      • [ ] There is an @CUSTOM directive in append/prepend position for the entire release notes as well as per sub-section therein.
    5. NOTE @CUSTOM directives may be useful for:
      1. next blocks which are edited in place frequently
      2. Refreshing release notes based on a new version of dripip that improves the layout somehow (e.g. adds a "Thanks to these users! ..." summary block)
      3. Perhaps/probably dripip has release notes style options sooner or later that users choose to adjust their settings on over time.
      • [ ] chore commits will not show up in the changelog.
      • [ ] feat commits will show up under Features section.
      • [ ] fix commits will show up under Fixes section.
      • [ ] All other commit types will show up under Improvements section.
      • [ ] Commits without commit types will show up in the changelog under Other Changes
      • [ ] Commits with BREAKING CHANGE will be copied to the BREAKING CHANGES section.
      • [ ] Commits from users not in the CODEOWNERS.txt file and not members of the org containing the repo will receive a special shoutout.
      • TODO Consider and/or being members on users in the same org as the repo
      • [ ] If a release section has no applicable commits then it is not rendered at all.
      • [ ] If this is a preview release then all commit entries are prefixed with next.<#>, the preview version they came from (we do this b/c pre-releases sum theirchanges since last stable)

    • [ ] Preview release release notes on GitHub Releases

        • [ ] Titled by the next version e.g. 1.2.0-next.5
        • [ ] Use the next git tag thus permitting a permalink
        • [ ] Every next release Updates-in-place the same GitHub release
        • [ ] Are marked as a pre-release
        • [ ] Contain the sum of changes since last stable
        • [ ] Formatted like so: 
          
Latest is <preview-version> released on <date>
Average time between stable releases is <x> days. Last one was on <date>.
      ``` 1. - [ ] When there is no `next` releases following latest stable release then the `next` GitHub release is formatted like: ``` No releases since last stable. ```

TODO

jasonkuhrt commented 5 years ago

Great catch by @janpio:

image

jasonkuhrt commented 5 years ago

Another principal:

Be judicious with developers' time. Stable releases should be batches of change. Users subscribing to previews opt-in for more noise. Everyone else, no.

jasonkuhrt commented 5 years ago

Spec Update

Weakky commented 5 years ago

Could you provide a link to the liberate cli? I can't find anything when googling it

jasonkuhrt commented 5 years ago

@Weakky it doesn't exist, it would be our invention. Was also thinking of libre.

jasonkuhrt commented 5 years ago

Been following this flow manually on https://github.com/prisma-labs/nexus-prisma/pull/434/files and its working ok. nexus pr release installed into nexus-prisma and then nexus-prisma pr release and then both installed into nexus-prisma example blog. Due to the node_modules file system trickery used linking doesn't "just work".

Examples of commands I am running:

❯ yarn publish --tag next --new-version "0.5.0-next.2"

❯ yarn publish --tag pr434 --no-git-tag-version --new-version "0.0.0-pr.434.$(git show -s head --format='%h')"
jasonkuhrt commented 5 years ago

Added TODOs for

jasonkuhrt commented 4 years ago

Spec Update

Came up with the following spec for release notes tonight. You can see below but it has been copied up to the opening comment too.

Release Notes ## Release Notes * Release notes will be added to a `CHANGELOG.md` file if present. * Release notes will be added to a `HISTORY.md` file if present. * Release notes will be added to a `README.md` `Changelog` section if present. * Release notes will be added to the GitHub release. * Release notes will be produced in Markdown format. * Release notes will be exportable as JSON. * Release notes for a single release will look like: ``` #### BREAKING CHANGES * [next.#] (link) [ty (link)!] * ... #### Features * [next.#] (link) [ty (link)!] * ... #### Patches * [next.#] (link) [ty (link)!] * ... ``` * `chore` commits will not show up in the changelog. * `feat` commits will show up under `Features` section. * All other commit types will show up under `Patches` section. * Commits without commit types will not show up in the changelog. * Commits with `BREAKING CHANGE` will be **copied** to the `BREAKING CHANGES` section. * Commits from users not in the [CODEOWNERS.txt](https://help.github.com/en/articles/about-code-owners) file will receive a special shoutout. * If a release section has no applicable commits then it is not rendered at all. * If this is a preview release then all commit entries are prefixed with `next.<#>`, the preview version they came from (we do this b/c pre-releases sum theirchanges since last stable) * Release notes on GitHub Releases * If the release is a preview then it is marked as a GitHub pre-release * preview releases will contain a sum of changes since last stable * Release notes in files * will have all `#` strings converted into links to the corresponding issue/pr. * will have the following template: ``` # Changelog ## `next` Latest is released on Average time between stable releases is days. Last one was on . ## (link to repo at SHA) ## ... ``` * preview releases will be grouped under `next` where a sum of changes since last stable accumulate. * When there is no `next` releases following latest stable release then the `next` section is formatted like: ``` ## `next` No releases since last stable. ``` * Release notes in `README.md` * will begin at the header level next from of that found for `Changelog` heading * Future features * It should be possible to manually change changelog files (for example with long form migration info under a breaking change) yet still accrue updates from automation.
janpio commented 4 years ago

Sound good. 2 notes:

  • All other commit types will show up under Patches section.

Wording might be problematic because of MAJOR.MINOR.PATCH. "Changes" maybe?

  • Commits without commit types will not show up in the changelog.

Maybe copy them to a "undefined" section by default that has to be cleaned out? Otherwise forgetting the type once might make something invisible.

jasonkuhrt commented 4 years ago

Wording might be problematic because of MAJOR.MINOR.PATCH. "Changes" maybe?

Agree. Its more consistent. For example feat: affects MINOR but we don't call the heading Minors.

Maybe copy them to a "undefined" section by default that has to be cleaned out? Otherwise forgetting the type once might make something invisible.

Thought about it. Wasn't sure. Can see that yes probably something has to be done. Some things we could do (none, some, all):

  1. List all undefined commits in terminal
  2. process exit 1
  3. emit all undefined commits to undefined-commits.md
  4. emit into file/gh-release-contents under undefined changes heading requiring user edits
  5. choose [safe?] defaults and support flags to opt-in/out of behaviours
jasonkuhrt commented 4 years ago

Spec Update

+ * `fix` commits will show up under `Fixes` section.
+ * All other commit types will show up under `Other Improvements` section.
pantharshit00 commented 4 years ago

Zeit has a similar release cli: https://github.com/zeit/release

Maybe we can take some inspiration

jasonkuhrt commented 4 years ago

Description Update

Adding support for:

+    * The `@META` `checksum` is derived from the generated release notes contents. Later on `liberate` can check contents against the checksum to easily detect drift (manual edits).
+    * The `@META` `liberateVersion` contains the same piece of data that would be in `packge.json` `version` snapshoted at the time of generation. This is useful to permit document format migrations from one version of `liberate` to the next.
+    * The `@CUSTOM` directives permit users to customize their release notes while preserving the ability for `liberate` to update them later on.
+    * There is an `@CUSTOM` directive in append/prepend position for the entire release notes as well as per sub-section therein.
+    * NOTE `@CUSTOM` directives may be useful for:
+      1. `next` blocks which are edited in place frequently
+     1. Refreshing release notes based on a new version of `liberate` that improves the layout somehow (e.g. adds a "Thanks to these users! ..." summary block)
+     1. Perhaps/probably `liberate` has release notes style options sooner or later that users choose to adjust their settings on over time.

Note that a robust spec for how to perform format migrations is not done and likely not to be before implementation work starts. The important thing is to have a reasonable foundation in place to make it possible later.

jasonkuhrt commented 4 years ago

Description Update

jasonkuhrt commented 4 years ago

Description Updated

One of our principals is

Releases should not show up as git commits, only tags.

This is because we care about CICD but also about a clean git history. If every commit is released (CICD) but releases require commits that means 50% of the git history is noise.

To solve this we figured out how to hoist version numbers out of [committed] source code and into git tags. However with release notes, as files, we have brought the problem of git commits back upon us. Worse yet, release notes are complicated and often require fine tuning because release notes are a kind of interface/content designed human consumption. Such fine tuning implies even more git commits, more than 50% of the git history. This is not workable.

Therefore the spec affordances for release note files will be removed. In turn there will be a focus on GitHub releases. In the future we may also consider GitHub wiki as another release notes target. Further future maybe pluggable targets.

jasonkuhrt commented 4 years ago

Description Update

Thought a bit more about naming. Liberate is somehow pompous or on the nose I feel, and not short making it functionally annoying to type on the command line. I considered a variety of alternatives but all the basic ones (cut scissor shear release ...) were taken on npm or had other problems, e.g.cut is a standard posix tool. I also felt going for something distinctive was worthwhile here. So going back to libre.

https://www.thoughtco.com/what-does-libre-mean-3079046

Instead, libre, related to words such as "liberate" and "liberty," usually refers to being free in the sense of being free of restraints or sometimes in the sense of being available.

This is a good metaphorical fit for this tool, since this tool is all about making work on a project easily available to users.

Like so many names, It is taken on npm too. But overall it is a distinct enough name that I'm going to give it a shot anyways. I considered a few publishing workarounds libre2 and librelibre. I think the latter is less evil than suffixing a 2. Numeric suffixes are unoriginal and suggest a continuity. Publishing under librelibre and stating the cli is exposed as just one utterance of the word seems intuitive enough.

janpio commented 4 years ago

https://github.com/prisma-labs/issues/issues/2#issue-498617600 contains all the current information if I just want to get an overview?

(Try to contact both @livingbreathing and npm, they might be able to help with the npm package name - repo seems to be gone and package unused)

jasonkuhrt commented 4 years ago

2 (comment) contains all the current information if I just want to get an overview?

Yep It is kept up to date with the latest thinking.

(Try to contact both @livingbreathing and npm, they might be able to help with the npm package name - repo seems to be gone and package unused)

Good point will do!

--update Email to Evan, the listed maintainer of libre, sent.

jasonkuhrt commented 4 years ago

I have updated the description with checkboxes. I am starting work over at https://github.com/prisma-labs/libre. I debated how to start tracking issues. My approach will be to use this as an epic to track the bulk of v1 development. I will not continue to put every new feature idea here though. Those will begin to get created as new issues on the libre repo. This issue has been useful to get the project off the ground, but was not intended to last forever.

So, as I make progress on libre I will come back here to update progress by simply checking the boxes off.

janpio commented 4 years ago

(Leaving a comment anyway will push this into my notifications, which might be beneficial - or not ;) )

jasonkuhrt commented 4 years ago

Thinking, rather than the --no-pr flag design, sub-commands.

$ libre preview           <-- automates selection between pr and trunk
$ libre preview pr
$ libre preview trunk

Possible alternatives to trunk: 

$ libre preview stable
$ libre preview master
$ libre preview latest
...
jasonkuhrt commented 4 years ago

Renamed from libre to dripip. Detail: https://github.com/prisma-labs/dripip/issues/13