[Discussion] Ideal Publishing Workflow

joshblack commented 5 years ago

After our retrospective for this last cycle, it became clear that we should revisit our release/publish workflow (particularly for bugs). This issue is to discuss what our ideal workflow/publish flow would be. We can then take the outcome of this and create action items to deliver against over the next release cycle 👍

Real quick background information on our previous process and existing process 👇

Previous process

Previously, we made use of an amazing tool called semantic-release. This allowed us to automatically publish packages for each of our repos whenever we merged in a Pull Request. This tool would detect what version change should occur by looking at the underlying commits from the Pull Request, and then would generate a release accordingly.

However, as our release cadence (and our team) grew, we ended up publishing too many releases and teams couldn't keep up. This left teams feeling distrustful over the stability of certain releases, and ultimately created a desire to pin carbon and not worry about keeping it up to date as each release jump could span multiple versions and subsequently carried risk for product teams

The as-is

As a result of the above, we may several adjustments to our release process. These include:

Hotfixes, or bug fixes, are grouped as-needed instead of released as merged
Minor releases are planned and delivered against, instead of being generated/dictated by commits into master
Major releases follow a similar model to Minor
We ship canary releases on every merge to master for the next minor release

Things that are painful right now:

This process is incredibly manual, outside of canary
Because the process is manually, collecting and documenting changelogs is harder
- We want to rely on conventional commits for this, but it keeps saying things should be a major change when they shouldn't

The to-be

This is where you come in! Drop down any ideas or suggestions to improve this process 🙏

joshblack commented 5 years ago

Would love to have an auto-generated changelog that we could point to for bug fixes. Also would love for manual parts of the process to be automated, and for conventional commits to be fixed in our project 😬

elizabethsjudd commented 5 years ago

@joshblack thanks for creating this issue! It was interesting to learn why you changed the process as someone who used to not update as often (especially with the number of major releases) but someone who also now struggles with the wait to get even my contributions from Carbon.

I wonder if a kind of happy medium approach could happen where the stable version of Carbon is released on a schedule but there was always an alpha version that did continuous delivery for users that do have the flexibility to pull in updates more on-demand. We are kind of doing something similar in WH as we are migrating to use Carbon 10 within PAL. Our biggest headache was keeping our "carbon-x" that controls the alpha release always in sync with our "master" but Scott came up with some clever CI code which automates that all for us now.

joshblack commented 5 years ago

@elizabethsjudd one thing that could help out with this is our canary builds. On merge to master we publish changed packages under the corresponding canary tag on NPM:

Although unfortunately this doesn't help in that CI case that you mentioned 😞 Just wanted to bring it up though 👍

elizabethsjudd commented 5 years ago

I was wondering what that was when I was looking at NPM yesterday! I was looking to see if the RC had been added and wasn't sure what the canary was.

mattrosno commented 5 years ago

Related to https://github.com/carbon-design-system/carbon/issues/2666, maybe we define how we use conventional commit types to reduce the number of feats that do minor version bumps?

And with that, whitelist conventional commit scopes to help with changelogs?

alisonjoseph commented 5 years ago

Would we be able to just automate PR's labeled as fix for . releases and anything labeled feat somehow gets held until the next minor release. Not sure how that would work though

joshblack commented 5 years ago

@alisonjoseph one constraint the existing model is operating under is that we try not to publish fixes as they come in and instead batch them up when appropriate just to reduce the amount of versions we ship (hotfixes are excluded from this rule). We can definitely remove this constraint, but that's why it's setup that way.

alisonjoseph commented 5 years ago

Yea, that makes sense, but it seems like we either automate it, with a bunch of fix releases, or we do it manually and people have to wait for their fix. Could we automate weekly releases w/ fixes bundled, would that still be too many?

joshblack commented 5 years ago

I bet we could 100% do weekly or biweekly (whatever cadence works out best) I think we could easily say that each week we publish a patch (maybe on Monday?)

scottnath commented 5 years ago

@joshblack I think the depth of this publishing workflow is great. We're looking forward to playing with multiple versions and running them through our CI as they get published. This is going to be especially helpful using Canary - so it's great that that exists!

question: is Canary every commit that gets merged into master?

expand commit messages

As to the commit messages turning into too many minor releases due to the prolific use of feat - I would suggest expanding on the commit message options. Then you'd have the option to more granularly choose which commits should end up in which release - and control which levels go into which semantic level.

FWIW, we use a subset of the gitmoji commit rules for our commit messages. Emoji-based commits can be combined to allow commit messages to label a commit as more than just one thing as well. Also, they're fun :rocket:!

commit messages decoupled from release rules

Right now, because our repo is still getting up to speed with Carbon X, we are keeping our version number under 1.0. We do this by adjusting the rules which semantic-release uses to make choices about the semantic release number it should create. So, in our case, breaking changes (:boom:) are set to make a minor release instead of a major one.

Point being...regardless of the needs of various release channels, the folks that are writing commit messages stick with the same rules and your CI process makes the choice of which commit msg types result in which release levels.

share commit message rules across monorepo and externally

Semantic-release also allows shareable configurations so that you can have one set of configs that work across all packages. An added benefit of that is that your configuration could be exported from the monorepo, and then external teams could use your both your commit-msg rules and your semantic-release rules so that their dev processes can more seamlessly sync with Carbon's.

new features of semantic-release allow granular releases

Lastly, the beta version of semantic release allows branch-specific releases. We're using this version in our releases, which allows us to release two separate npm modules automatically.

joshblack commented 5 years ago

@scottnath

question: is Canary every commit that gets merged into master?

Yup!

As to the commit messages turning into too many minor releases due to the prolific use of feat - I would suggest expanding on the commit message options. Then you'd have the option to more granularly choose which commits should end up in which release - and control which levels go into which semantic level.

I think our preference would be lean on conventional commits and try and figure out a process change versus diverging from the default format. If this ends up biting us, then definitely open to changing it haha.

Point being...regardless of the needs of various release channels, the folks that are writing commit messages stick with the same rules and your CI process makes the choice of which commit msg types result in which release levels.

Our big thing is that we want to throttle releases, operating under the constraint that each release we ship introduces design and technical debt across the company and so we should minimize the amount of changes in our system and subsequently limit the number of times teams need to upgrade.

Lastly, the beta version of semantic release allows branch-specific releases. We're using this version in our releases, which allows us to release two separate npm modules automatically.

Unfortunately a big part of this release shift has been moving away from branch-specific releases. A big part of this strategy comes from the babel/react teams sharing insights that aspects of this model may not have worked for their use-case: https://reactjs.org/blog/2017/12/15/improving-the-repository-infrastructure.html#branching-strategy

I think conceptually having master be latest code that corresponds with the next minor release makes sense, where this model can feel awkward is when we have to explicitly break off for patch releases 😞

While I absolutely love the tooling and support that semantic release brings to the table, unfortunately some of the default/assumed behavior or options to configure can conflict with some of the design goals of this release process.

scottnath commented 5 years ago

I think our preference would be lean on conventional commits

Totally figured! Threw out the gitmojis because of the depth of options. The main point I meant to make was that more granular commit message options could help with releasing specific code.

Whatever choice Carbon makes, it would be fantastic if the rules for commit messages, and if possible the code which enforces the message format, were exported for use in other systems. That way teams like PAL that want to more closely align with Carbon can also align in how we do the day-to-day tasks like writing commit messages or the way we write code (enforced by shared linting rules.) All with the goal of making it more seamless to contribute because we work so similarly.

joshblack commented 5 years ago

Totally figured! Threw out the gitmojis because of the depth of options. The main point I meant to make was that more granular commit message options could help with releasing specific code.

Sorry, wasn't meant as a comments towards the gitmoji stuff (totally remember that setup from before with Sam and thought it was fun 😄) just was directed towards switching up the commit message semantics. The problem we ran into was that our usage of feat or fix mapped 1:1 with a release and instead we wanted some kind of batched heuristic, which branches can totally solve.

With respect to the rules for commit messages, we can definitely try. Ultimately we use @commitlint/config-conventional OOTB with a couple of deviations.

stale[bot] commented 5 years ago

We've marked this issue as stale because there hasn't been any activity for a couple of weeks. If there's no further activity on this issue in the next three days then we'll close it. You can keep the conversation going with just a short comment. Thanks for your contributions.

stale[bot] commented 5 years ago

As there's been no activity since this issue was marked as stale, we are auto-closing it.

carbon-design-system / carbon