7rulnik
commented
4 years ago @arcanis maybe we should add case study label to research problems with current usage of changesets?
Open 7rulnik opened 4 years ago
7rulnik
commented
4 years ago @arcanis maybe we should add case study label to research problems with current usage of changesets?
arcanis
commented
4 years ago Changelog would be awesome - even for this very repository we would love this feature! So far however time has been short to implement it, so if someone could help us on that, it would be amazing š
The way I see it, we cannot resort on conventional commits. They are fine when managing a single project, but they don't compose very well with the model used by version check - we'd need to find out the original commit that led to a particular release file, and that can be tricky and error-prone.
Instead, I'd like to add an input box (nothing fancy, just a single line, maybe?) that would be stored within the release file along with the rest of the data. Then, when applying the versions, the stored changelog lines would be each applied to all related workspaces and aggregated into a markdown output (which would be obtained with yarn changeset changelog?).
Note that we would treat direct releases pretty much the same way as transitive releases for the purpose of the changelog generation. So for example, if we make a change into plugin-git that require to re-release the CLI as well, here's what could be generated:
yarnpkg/cli
- Add support for foo By username
yarnpkg/plugin-git
- Add support for foo By username
JureSotosek
commented
4 years ago So this is how I imagined this working:
version commandWe add a --changelog flag.
When you call the command with that flag, you are prompted to enter a short description of the bump. That description is then be added to changelogs of all the workspaces where the version has been bumped.
version check commandWe add a --changelog flag.
When you call the non-interactive command with that flag, it not only checks that versions were bumped, but it also checks that all workspaces that were bumped contain a short description of the bump.
When you call the interactive command with that flag, you are not only be prompted to bump versions, but there is also a text input present, where you enter a short description.
The text input is only be present for direct releases, the workspaces that depend on the direct releases just copy the descriptions of the corresponding direct releases. How @arcanis described it above.
The descriptions are then added to the release files.
version apply commandThe command works the same as before, except now, it also appends the short descriptions to the corresponding changelogs.
There is a changelogs field added, that contains the short descriptions for all bumped workspaces.
So for example, if you change something in the yarnpkg-libui, bump its version, add a short description, and then also bump the versions of plugin-version and plugin-interactive-tools, since they depend on yarnpkg-libui, the release file would look something like this.
releases:
"@yarnpkg/libui": prerelease
"@yarnpkg/plugin-interactive-tools": prerelease
"@yarnpkg/plugin-version": prerelease
changelogs:
"@yarnpkg/libui": "Increased spacing in the item's component."
"@yarnpkg/plugin-interactive-tools": "Increased spacing in the item's component."
"@yarnpkg/plugin-version": "Increased spacing in the item's component."This then results in all of the three packages getting the description added to their changelogs.
An example of how an entry into the changelog would look like, very simple for now:
Do we also want to keep a root changelog, that would keep track of all of the version bumps in all of the workspaces? I'm not sure about this one, maybe it can wait until the next iteration of this feature.
What do you guys think?
7rulnik
commented
4 years ago Let's take a look at changesets.
One of the main ideas that you can read changelog entries easily and edit it later. So I believe that markdown is better than YAML it this case. Also, changelog entry can contain many text and code blocks for example.
Note that changelog entries have some human-readable names like rich-melons-press.md so it's easier to navigate between them and review changes or edit it.
As input, I would expect that command will open my preferable editor/IDE via $EDITOR variable because maybe I would write several paragraphs of text and also I want to see a preview for written markdown. Maybe it could be toggle via option?
Also it will be great (and important?) to not break backward compatibility. Can we support both markdown and yaml formats? Or maybe as workaround we can link markdown files in yaml? Something like that (but I don't really like it):
releases:
"@yarnpkg/libui": prerelease
"@yarnpkg/plugin-interactive-tools": prerelease
"@yarnpkg/plugin-version": prerelease
changelogs:
"@yarnpkg/libui": "./some-file.md"
"@yarnpkg/plugin-interactive-tools": "./some-file.md"
"@yarnpkg/plugin-version": "./some-file.md"Also there are could be more that one changelog entries for one PR.
More details here: https://github.com/atlassian/changesets/blob/master/docs/detailed-explanation.md
I think we should have different changelog drivers: we have github, bitbucket, gitlab at least.
To be honest I feel weird that we are discussing features that already implemented in changesets so maybe better to find a way how we can integrate it?
arcanis
commented
4 years ago
versioncommand
The --changelog option would only be applied when using --deferred, right? Otherwise where would we keep it, since there would be no YAML file?
version applycommand
I think it might be interesting to ignore version apply for this iteration. Instead, I'd like to introduce a new command, yarn changeset changelog, that would generate a Markdown output based on the content (but without changing anything to the repo - it would be a side-effect-free command. So for example, the typical release workflow would do this:
yarn changeset changelog >> CHANGELOG.md
yarn changeset applyDoing this would be quite handy, because it would become possible to generate the changelog at any given point without mutating the repository. For example, let's say I want to quickly look at what's going to be released, I can just run yarn changeset changlog and everything is here.
Do we also want to keep a root changelog, that would keep track of all of the version bumps in all of the workspaces? I'm not sure about this one, maybe it can wait until the next iteration of this feature.
With the yarn changeset changelog, the responsibility of actually generating the changelog would fall to the user, so this problem wouldn't be there.
Example changelog
I suggest to also find the merge commit that added version files, and to link to it (we can grep the trailing (#XXX) from the merge commit message to derive the PR number) + their authors. For example, an entry could be:
@yarnpkg/monorepo
One of the main ideas that you can read changelog entries easily and edit it later. So I believe that markdown is better than YAML it this case. Also, changelog entry can contain many text and code blocks for example.
There's no reason why Markdown cannot be stored within a YAML file:
changelog: |
Fixes our repository constraints *(warning: experimental)*As input, I would expect that command will open my preferable editor/IDE via $EDITOR variable because maybe I would write several paragraphs of text and also I want to see a preview for written markdown. Maybe it could be toggle via option?
That's a good point - perhaps we could follow the Git model and make yarn version --deferred open the EDITOR, and yarn version --deferred -m "Foobar" add the changelog entry from the command line without opening the editor?
Also there are could be more that one changelog entries for one PR.
Perhaps in a followup, but it may make the UI confusing. I prefer the initial version to simply cover all the workspaces, and we can go from here to figure out whether this model should be refined.
I think we should have different changelog drivers: we have github, bitbucket, gitlab at least.
Too complex for the first iteration. We don't need driver for now, at best we only need to find the PR number and generate a hashlink from it (all providers are assumed to follow this standard), with perhaps a fallback to instead print the short commit id when there's no PR number to be found in the merge message.
To be honest I feel weird that we are discussing features that already implemented in changesets
The version workflow was designed without being aware of the Changeset project. While there's certainly value in taking a look at what exists elsewhere, I think ultimately we're in a unique position of implementing a workflow that's integrated deeper than what third-party projects could do. At the very least, it's worth considering alternatives š
abdes
commented
4 years ago Another way to do this would be to export proper hooks from the version plugin so that someone else can write the plugin that handles change logs properly.
Please also see the issue logged for changesets to properly work with yarn v2 at: https://github.com/atlassian/changesets/issues/432 for reference
abdes
commented
4 years ago To be honest I feel weird that we are discussing features that already implemented in changesets
The version workflow was designed without being aware of the Changeset project. While there's certainly value in taking a look at what exists elsewhere, I think ultimately we're in a unique position of implementing a workflow that's integrated deeper than what third-party projects could do. At the very least, it's worth considering alternatives š
I think that the statement that 'integrated deeper' means better than third-party is a bit extreme and dangerous. One of the best features of berry is the introduction of plugins and hooks so that integration with yarn does not mean a new release of yarn. It's actually pretty desirable that an endorsed plugin by yarn (i.e. version) also follows that direction and exports the proper hooks for a third-party project to integrate deeper with yarn. If that is deemed too much, then would you please consider enhancing the workflow advocated by version to also include pre-release like changesets, generating CHANGELOG.md like changesets or better, etc...?
Andarist
commented
4 years ago Hi there š I'm one of the maintainers of Changesets, so I thought I would pitch in here as @abdes has pointed me to this thread.
Let's start with a few comments.
The text input is only be present for direct releases, the workspaces that depend on the direct releases just copy the descriptions of the corresponding direct releases. How @arcanis described it above.
This IMHO is not enough. Changelogs are supposed to be consumed primarily by humans - they are informative. Even in simple setups, this could easily create confusing entries. There are two scenarios to consider here:
node_modules when upgrading your dependencies in a consuming project.Release Files format
&
There's no reason why Markdown cannot be stored within a YAML file
Yes, technically you could store Markdown in YAML but I would consider this to be less user-friendly overall. You won't get the same support from editors in editing this. If you keep everything in a single YAML then it would also probably be more prone to git conflicts.
As @7rulnik has mentioned, this is vastly important for Changesets:
One of the main ideas that you can read changelog entries easily and edit it later. So I believe that markdown is better than YAML it this case. Also, changelog entry can contain many text and code blocks for example. And I believe is one of the reasons behind Changesets getting traction and love from their users.
Do we also want to keep a root changelog, that would keep track of all of the version bumps in all of the workspaces? I'm not sure about this one, maybe it can wait until the next iteration of this feature.
This has been requested at Changesets, but so far we haven't implemented support for this. There are many workflows that people are using right now and it's hard to satisfy them all, we also try to not overcomplicate stuff in the long run, so additional features like this are considered carefully. We don't want to rush into implementing them right way.
Also there are could be more that one changelog entries for one PR.
This is also an important "feature" of Changesets. Support for this enabled user-friendly changelogs from PRs that touch multiple packages - which are not rare in monorepos.
Perhaps in a followup, but it may make the UI confusing. I prefer the initial version to simply cover all the workspaces, and we can go from here to figure out whether this model should be refined.
@arcanis has commented this about the former thing - multiple changelog entries for a single PR. I'm not sure exactly how do you envision this to work in Yarn - but in Changesets this works already and I haven't heard any DX complaints about it. We have an interactive CLI command, with autocomplete and all that jazz that guides users through the process.
Doing this would be quite handy, because it would become possible to generate the changelog at any given point without mutating the repository. For example, let's say I want to quickly look at what's going to be released, I can just run yarn changeset changlog and everything is here.
This I don't quite understand. Could you sum up the complete workflow that has been discussed so far? How could you not mutate the repository and implement deferred publishes at the same time? š¤
The version workflow was designed without being aware of the Changeset project. While there's certainly value in taking a look at what exists elsewhere, I think ultimately we're in a unique position of implementing a workflow that's integrated deeper than what third-party projects could do. At the very least, it's worth considering alternatives š
You are, of course, allowed to implementing your own workflow - but as people have already said: the overlap between this and Changesets is just huge. As @abdes said, given your plugins architecture, it really sounds like you should offload this to 3rd party tooling. Maybe provide some dedicated wrappers for them in here. Giving away some of the maintenance to others wouldn't also hurt Yarn - it already has a huge and important responsibility, I don't particularly think that packing even more into Berry would be that beneficial to you. Yes - you would gain control, so maybe that's important enough for you to do the hard work on your own. We could cooperate more closely to work out a shared solution though, setting up integration tests and all.
Changesets - while working and serving its purpose - still have some minor gaps, there are hard answers with no ideal answers. We have already figured out a lot though and it would be a shame to reinvent this workflow again.
arcanis
commented
4 years ago Hey @Andarist! Thanks for shimming here with such a detailed comment! One thing that's important enough to be answered first:
You are, of course, allowed to implementing your own workflow - but as people have already said: the overlap between this and Changesets is just huge. As @abdes said, given your plugins architecture, it really sounds like you should offload this to 3rd party tooling.
I actually really agree - when I started working on the changeset feature I didn't look enough at existing solutions, and missed your project until @zkochan pointed it to me š Had I noticed it, there's a good chance I'd have at least tried to integrate both tools. If you're open to start a discussion to discuss what that would look like, I'd be very interested!
Could you sum up the complete workflow that has been discussed so far? How could you not mutate the repository and implement deferred publishes at the same time?
Updates can be applied "in memory". For example, our current plugin mutates all the workspace instance in memory, and only during the following install are they persisted on the disk. It would be easy to completely skip the install, and instead print a summary of the changes that would have been applied.
Yes, technically you could store Markdown in YAML but I would consider this to be less user-friendly overall. You won't get the same support from editors in editing this. If you keep everything in a single YAML then it would also probably be more prone to git conflicts.
I see you're using Frontmatter - that's a nice solution indeed š How does that work however when you wish to provide upgrade information on multiple packages, like you mentioned before? Do you detect which workspace a particular paragraph applies to per the section title and aggregate them all?
If you wish, perhaps we can make a "meeting" on Discord this week to discuss this in a more interactive way?
abdes
commented
4 years ago I love OpenSource :-), Thanks @arcanis and @Andarist š .
zkochan
commented
4 years ago Please include me in your future discussions as well. We use changesets+pnpm in my company of 200+ devs. Also, I recommend everyone who uses pnpm to use changesets for publishing. So I'd like changesets to be the de facto standard. The only issue with it is the crazy lack of documentation.
Andarist
commented
4 years ago If you're open to start a discussion to discuss what that would look like, I'd be very interested!
I would be interested in that š Feel free to reach out to me any time, I'm not sure if you have time to work on this right now. I might also not give this 100% of my attention this week - but I will do my best to answer any potential questions and brainstorm a little bit in the asynchronous fashion.
If you wish, perhaps we can make a "meeting" on Discord this week to discuss this in a more interactive way?
I possibly could do this next week or later.
I see you're using Frontmatter - that's a nice solution indeed š How does that work however when you wish to provide upgrade information on multiple packages, like you mentioned before? Do you detect which workspace a particular paragraph applies to per the section title and aggregate them all?
When running a CLI command we list all "changed" packages at the top and the rest below them, one can select packages for which they'd like to create a changeset file (that will become a changelog entry later), after that, we ask for bump types for each package (or rather - we ask which should be major bumped, which minor bumped and leftovers are considered as patch, so u get at most 2 questions at this step of the flow, from what I recall). And as a final step, we ask for the actual message describing the change.
So such a single yarn changeset run results in a single changeset file containing release intents for N packages (they can have different bump types as described above) but with a single message. If you want to create different messages you just have to run this multiple times. CLI is nicely guiding a user through the process, but changeset files can be created manually when one gets accustomed to the flow and would like to speed things up for themselves - they are just simple markdown files with a frontmatter after all.
I love OpenSource :-), Thanks @arcanis and @Andarist š .
Me too! ā¤ļø
So I'd like changesets to be the de facto standard. The only issue with it is the crazy lack of documentation.
We could definitely try to improve this. If you have any particular things that you feel are especially poorly documented - please open issues about them and we'll try to provide docs covering those topics.
arcanis
commented
4 years ago So such a single
yarn changesetrun results in a single changeset file containing release intents for N packages (they can have different bump types as described above) but with a single message. If you want to create different messages you just have to run this multiple times.
I see - I think those are the differences with our current workflow:
Calling the command repeatedly will always use the changeset file that's already in the current branch. I find that handy, because you sometimes want to amend the changes you've made.
We only print the dependent packages from those who will get released, rather than all of them (we use Ink to update the display depending on the previously selected options).
We show a tabular interface asking to select a type of release for each row. This way there are more choices (we also handle prereleases, for example).
In particular, at the moment, declining to bump a package is possible (for example if you only changed the README), but it's an explicit choice (like minor, major, etc). It's a bit annoying at the moment, we want to simplify that.
I possibly could do this next week or later.
Great, I'll setup something š
arcanis
commented
4 years ago Oh and, one problem that we currently have, it's sometimes a bit difficult for external contributors to know what to bump (for example that modifying the plugins require to mark the CLI package for release). I'd like to eventually fix that - perhaps with "presets" in the UI, or with a GH action that would allow us to set those fields from the PR interface.
abdes
commented
4 years ago Oh and, one problem that we currently have, it's sometimes a bit difficult for external contributors to know what to bump (for example that modifying the plugins require to mark the CLI package for release). I'd like to eventually fix that - perhaps with "presets" in the UI, or with a GH action that would allow us to set those fields from the PR interface.
@arcanis, that actually may be more general than just during the version bump. Today yarn does a good job in identifying inter-workspace dependencies, so we can definitely implement a command in the plugins to for example identify all affected packages by a change. This is similar to what Nx does with nx affected command or to what rush does with --from or --to options to select a package and its dependents or its dependencies. Nx solves the implicit or manual dependencies through the workspaces config. You can specify that a certain workspace depends on another although that dependency is not detected by yarn. If you do so, then the affected algorithm will take it into account.
Actually such feature is very useful to for example:
run yarn npm publish on a workspace and its dependents instead of today run on all workspaces and silently fail on the non-bumped ones,
run tests on affected workspaces,
set an implicit dependency on tsconfig.base.json and if that file changes all TS workspaces get rebuilt,
...
Andarist
commented
4 years ago Calling the command repeatedly will always use the changeset file that's already in the current branch. I find that handy, because you sometimes want to amend the changes you've made.
You can amend changes just fine using our workflow - changeset files are physically written to disk so it's super easy to add more, remove some or edit some. I might still fundamentally misunderstand how your workflow looks like though and maybe that's why there is a small misunderstanding in this area between us.
Using full Changesets workflow you create changesets at any time (even after your PR get gets merged into the base branch). When changesets land on a base branch our github action (that consumers should have configured for their repo) creates a versioning PR like this: https://github.com/emotion-js/emotion/pull/1963 . And only when this gets merged to the base branch actual releasing happens - so releases are not 100% automated, but it's specifically designed for people to have control over the releasing. It's mostly automated in this regard. So to rephrase this a little bit - changeset files sit in the repo, written to disk, waiting for the release to happen.
Could you give a brief overview of how this works right now in your case? Or point me to some docs?
We only print the dependent packages from those who will get released, rather than all of them (we use Ink to update the display depending on the previously selected options).
Hm, this doesn't sound much different from us. Maybe just in some details - but the basic idea sounds the same, show people what we think should be shown to them to minimize the cognitive load and speed up things for them.
We show a tabular interface asking to select a type of release for each row. This way there are more choices (we also handle prereleases, for example).
Would have to take a look at this, but the tabular interface sounds like a cosmetic difference. It should be possible to build any interactive CLI one could imagine for this - given building blocks we provide (or which we could provide, not sure if everything is exposed right now for public usage).
As to prereleases - we handle this as well, but probably in a different way than you do. Prereleases are a complex topic of its own so we can postpone discussing this until later.
In particular, at the moment, declining to bump a package is possible (for example if you only changed the README), but it's an explicit choice (like minor, major, etc). It's a bit annoying at the moment, we want to simplify that.
We also support "none" bump type that can be used for this. I don't think though that there is any way to enforce changesets for all affected packages right now - one would have to build this as a custom GitHub action (and I think people have already done that because I recall reading about such workflows in the issues).
Oh and, one problem that we currently have, it's sometimes a bit difficult for external contributors to know what to bump (for example that modifying the plugins require to mark the CLI package for release). I'd like to eventually fix that - perhaps with "presets" in the UI, or with a GH action that would allow us to set those fields from the PR interface.
In changesets if CLI depends on plugins and you add a changeset for the plugins package then CLI will be bumped at least with a patch bump and the updated dependencies will be listed in its changelog. There is no way to enforce (right now) that one has to write a changelog entry for CLI though.
We also have a "linked" packages concept - in this case, the CLI would be bumped to the same version as plugins package. This is similar - but not the same (!) - to Lerna's fixed versioning. The difference is that if you have 2 linked packages that do not depend on each other (let's say A@1.0.0 and B@1.0.0) and you minor version B then you will end up with A@1.0.0 and B@1.1.0, but if you now patch version A then you will end up with A@1.1.1 and B@1.1.0 - whereas with fixed versioning you would get 1.1.0 for both the first time and 1.1.1 for both the second time.
As to the PR interface - we provide a bot which makes it easier for maintainers to add changesets directly from GitHub (assuming PR author has allowed maintainers to commit to their fork). It is done by preparing a URL such as this: https://github.com/hudecsamuel/xstate/new/state-value-type?filename=.changeset/smooth-feet-learn.md&value=---%0A%22%40xstate%2Ffsm%22%3A%20patch%0A---%0A%0Aupdated%20State%20interface%20value%20type%0A
SimenB
commented
3 years ago Not to be that person, but any updates on this? Lerna doesn't seem like it's actively maintained anymore, so I'd love to remove the dependency and just rely on yarn for publishing š
arcanis
commented
3 years ago We discussed with @Andarist and I think the takeaway was that we would want to uniformize the syntax used by Yarn and Changesets so that we'd still be able to keep the UI interface we currently have (in particular where multiple workspaces can be affected to different release strategies, or none at all), but would also be able to leverage the Changesets tools.
At the moment however it hasn't yet been acted upon, so the best approach would be to use the regular release workflow, although it's not exactly a 1:1 mapping with Lerna. For what it's worth, Babel has removed Lerna at the same time they migrated to Yarn 2, in part by implementing a plugin to customize the behavior to exactly what they were after - perhaps that would be an option?
SimenB
commented
3 years ago Thanks for the update @arcanis! Writing a custom plugin sounds a bit much for evening hacking, but I'll take a look. Maybe @nicolo-ribaudo wants to share his thoughts in this issue as well? š
akphi
commented
3 years ago @SimenB I learned a lot from the way you guys work on https://github.com/facebook/jest
Could you kindly share with me what exactly is preventing you from removing the lerna at this point? I doubt it's this changelog feature that you are waiting for right? - because as I can see, for each PR, you manually add to CHANGELOG.md file. So is it something about the release/publish process that makes you stick with lerna?
I also took some time to checkout changesets. At its core, it brings about 2 core ideas:
This is exactly what Jest already did. But
changesetsmakes it a bit more convenient by splitting the changelog into oneCHANGELOG.mdfile at each package rather than maintaining a global changelog likebabelorjest. Unlikebabelwhich useslerna-changelogto generate their changelog usingconventional-commit,changesetsis like Jest, relies solely on user input to create the changelog.
yarn version plugin already covered super well.@arcanis I reckon this might not be the best place to ask, but tools like changesets and lerna help with creating github tags (like @namespace/pkg1@1.0.0). Basically, this can totally be done with a separate script for now, which I see you are already doing it
https://github.com/yarnpkg/berry/blob/master/scripts/release/01-release-tags.sh
Do you think this kind of feature will make it to the version plugin?
maticzav
commented
3 years ago Hi everyone, thank you for working on this! I (sadly) have to second @SimenB - any progress on this?
For anyone looking for a workaround though, I have a working setup that uses changeset with Yarn 2.0.
changeset for changelog generation and versioning,yarn with yarn workspaces as a package manager.The main problem I had was that changeset wouldn't strip workspaces:* prefix from my dependencies and I had to rely on yarn npm publish.
All in all, here's the code:
yarn changespace and fills out the form,The build script is primarily inspired by Jest's TypeScript build script that they use(d).
PS.: @arcanis I can probably help; +1 on the plugin approach - I don't think we should reinvent the wheel; do you have an RFC process set-up for Yarn Berry? It seems like we could start with an RFC and get everything discussed so far outlines in one place.
PS.: Sorry for the emojis, I usually skim through such long threads and wish somebody made their solution a tiny bit more visible.
akphi
commented
3 years ago @maticzav using your approach, git tags will no longer be created.
If we don't use workspace:* and use explicit versions for workspace dependencies instead, changesets will bump those specific versions properly during version phase.
After looking at the code for their github-actions https://github.com/changesets/action/blob/master/src/run.ts#L88
Seems to me that to make things like auto-creating GitHub release and adding tags, what you need to do is to console log out the right messages.
arcanis
commented
3 years ago I don't think we should reinvent the wheel; do you have an RFC process set-up for Yarn Berry? It seems like we could start with an RFC and get everything discussed so far outlines in one place.
We don't really use formal RFCs outside of very core things - from my experience RFCs rarely lead to PRs, so the best way is to start directly from a PR, and ask questions on our Discord channel if you're unsure about what should be done. This way we can quickly iterate š
In this particular case as I mentioned we really want to support changelogs, it's just that it's not one of our top priorities at the moment. But if someone like you can lend a hand, I'm sure we can make quick progress! The plan would be similar to this:
Change the generated version files to be closer from what Changeset generates. In particular, switch to markdown (but no need for a full markdown parser, I think it would be overkill). This will allow us to share more tools between our projects, which would let us use their GitHub bot, etc.
Implement an input box in yarn version check -i so that users can explain what their change is about. For the prototype an single input box is enough. Later on we may want one input per workspace, but not in the first iteration.
Generate changelog out of the existing files. This should be exposed as 1/ a function in versionUtils.ts, 2/ a yarn version changelog command, 3/ via the yarn version apply command (in this last case the changelog would be inserted at the top of the CHANGELOG.md file).
I think with that we'll have enough to start experimenting and see what we like and what we want to change. If someone could implement these three points it would be invaluable. I'd definitely send you one of our Yarn tshirts š
arcanis
commented
3 years ago tools like
changesetsandlernahelp with creating github tags (like@namespace/pkg1@1.0.0). Basically, this can totally be done with a separate script for now, which I see you are already doing it
Yes, I think this would make sense. I didn't implement it because at the time Yarn was trying to be CVS-agnostic (I was at FB, and FB used Mercurial), but nowadays I think it makes sense to rip the bandaid and exclusively offer Git-oriented features. If someone wants another CVS they can maintain their own plugins.
If you make a PR adding autotags support (note that this includes making sure that it integrates properly with our scripts! since we really want to dogfood the features we offer) I'd gladly review and merge it.
akphi
commented
3 years ago
- Change the generated version files to be closer from what Changeset generates. In particular, switch to markdown (but no need for a full markdown parser, I think it would be overkill). This will allow us to share more tools between our projects, which would let us use their GitHub bot, etc.
@arcanis Isn't this going to be a breaking change to the version plugin?
arcanis
commented
3 years ago Yes and no, the best would be to support reading both old and new format, and only generate new files with the new format. Differentiating the two at read time won't be too difficult since they'll have different file extensions.
Also, we've just started work on Yarn 3 on the master branch so it's the perfect time to land this kind of changes š
akphi
commented
3 years ago @arcanis nice! I can't promise much but I can try to give it a shot during the weekend. This makes me wonder though, if we use their format, if only they expose their parser and logic for building the changelog, we can reuse those, otherwise we end up with duplicated code in both code base. Maybe this amount of code is minimal, maybe it's not, if it's not, should we expose these methods to handle changesets version info markdown from @atlassian/changsets?
What do you guys think @Andarist @arcanis?
@arcanis: having a dependency on changesets might not be great, I don't know how you feel about this š
akphi
commented
3 years ago 3. Generate changelog out of the existing files. This should be exposed as 1/ a function in
versionUtils.ts, 2/ ayarn version changelogcommand, 3/ via theyarn version applycommand (in this last case the changelog would be inserted at the top of theCHANGELOG.mdfile).
Also, regarding this, what is the desired behavior then. changesets add this to individual workspace CHANGELOG.md. Since their change info file looks like this:
pkg1: patch
pk2: minor
Some textThey will bring Some text to both CHANGELOG.md of pkg1 and pkg2. If we collate the change on the top-level CHANGELOG.md probably we have to scope it to produce one line like:
[pkg1, pk2] Some text(similar to behavior of lerna-changelog, which is used by babel - https://github.com/babel/babel/blob/main/CHANGELOG.md
But the more I think about this the more it makes less sense than keeping them in separate CHANGELOG.md files, because if you keep them in the top-level CHANGELOG.md file, we have no idea which version this change is under.
Libraries like Jest and babel tends to keep moving their version together so in their top-level CHANGELOG.md they can afford to do this.
I really don't know what is the best behavior here, do you have any thought on this @arcanis ?
arcanis
commented
3 years ago This makes me wonder though, if we use their format, if only they expose their parser and logic for building the changelog, we can reuse those, otherwise we end up with duplicated code in both code base.
The parsing is likely simple enough to be done without depending on anything (frontmatter is literally just yaml between two indicators; a simple regex should be enough), but perhaps the changelog generation would be worth sharing, yes. I don't know how feasible this is though, so I would see that as a followup.
But the more I think about this the more it makes less sense than keeping them in separate CHANGELOG.md files, because if you keep them in the top-level CHANGELOG.md file, we have no idea which version this change is under.
Note that for the first iteration it's really fine if we support only one log entry that gets used for all packages marked inside a single version file. With that being said, my idea was to use headers. So for instance, let's imagine the following file:
---
"@yarnpkg/cli": minor
"@yarnpkg/core": minor
---
Does something newThen "Does something new" is associated to both cli and core (how exactly it's rendered is up to discussion). On the other hand, if I write this:
---
"@yarnpkg/cli": minor
"@yarnpkg/core": minor
---
## @yarnpkg/cli
Does something newThen only cli has a changelog entry, not core. And finally, if I write this:
---
"@yarnpkg/cli": minor
"@yarnpkg/core": minor
---
## @yarnpkg/cli
Does something new
## @yarnpkg/core
Some API fixThen cli and core will have different changelog entries. Again, how it's rendered isn't fixed, and I expect it to be improved over time.
This way I believe we would be compatible with Changesets on the default behavior, while remaining open to more complex changelog generation where different packages have different entries. How does that sound?
akphi
commented
3 years ago @arcanis sounds good to me!
maticzav
commented
3 years ago @arcanis @akphi apart from the versioning step, what is the desired publishing workflow for workspaces? I understand that there's lerna, but lerna doesn't work all that well with Yarn 2.0 and we already have a fantastic versioning tool.
What would be the link between Yarn and versioning tools that trigger the publish?
Andarist
commented
3 years ago Well, we assume that the whole content of a changeset land as a changelog entry - one is free to use the whole markdown to describe the change. If you plan on using ## as some kind of delimiter then this makes headers impossible or requires special logic for headers containing a package name. In Changesets, one would just create 2 changeset files for this
---
"@yarnpkg/cli": minor
---
Does something new---
"@yarnpkg/core": minor
---
Some API fixAlso, note that creating a changeset for the dependency package will often lead to bumping its dependants with a patch version. It keeps things up to date but you don't always have to specify version bumps manually for everything.
As to changelog generators - we support custom modules for this, so if you could live with the same API as we have (or propose changes to it) then changelog generators could be reused between both.
arcanis
commented
3 years ago If you plan on using
##as some kind of delimiter then this makes headers impossible or requires special logic for headers containing a package name.
Do you have an example of a changelog entry with a header? From my experience we've only used one-liners as changelog entries, I'm curious to see what it would look like with multiple paragraphs.
I think keeping a single file is a requirement for us, but perhaps we can use a different separator. For instance we could do just like you did, but in a single file:
---
"@yarnpkg/cli": minor
---
Does something new
---
"@yarnpkg/core": minor
---
Some API fixDo you have an example of a changelog entry with a header? From my experience we've only used one-liners as changelog entries, I'm curious to see what it would look like with multiple paragraphs.
For example this one - https://github.com/davidkpiano/xstate/releases/tag/%40xstate%2Finspect%400.4.0 . While single paragraphs are the usual case I find it very helpful (for users!) to provide short code snippets, even images, or multiple paragraphs from time to time.
I'm a fan of simple rules and "here is a frontmatter and the rest is just markdown" is a simple rule - no funky exceptions.
I think keeping a single file is a requirement for us
Why is that? I'm genuinely curious because it sounds like a matter of taste/personal preference and not a technical limitation. I don't mind much if it's that - just I wouldn't probably call it a hard requirement then.
arcanis
commented
3 years ago Why is that? I'm genuinely curious because it sounds like a matter of taste/personal preference and not a technical limitation. I don't mind much if it's that - just I wouldn't probably call it a hard requirement then.
I don't remember if Changeset supports that as well, but running yarn version check -i twice will not create two files - it'll edit the current one. This is important, because we sometimes want to edit the choices we've already made (for instance I do this semi-often when making changes to external contributors' PRs).
To achieve that, when running the command, we first check if the changed files include one file inside the version file directory. If there is then we parse it to get the initial UI state, and store any modification in-place. If we were to support multiple files, it'd make the logic quite more complex to reconcile them all, and would make merges ambiguous.
And as you mention there's also that I'd prefer to keep a 1 PR = 1 file mapping. I find it useful in our repository, to get a quick overview of every PR that's going to be released, with no duplicates:
Andarist
commented
3 years ago I don't remember if Changeset supports that as well, but running yarn version check -i twice will not create two files - it'll edit the current one.
We always create a new one.
This is important, because we sometimes want to edit the choices we've already made (for instance I do this semi-often when making changes to external contributors' PRs).
I do that quite often as well - by just editing the existing .md file.
To achieve that, when running the command, we first check if the changed files include one file inside the version file directory.
How do you determine that? Looks like generated filenames are based on git commits but given how people can do a lot of things with their living branches (amending, force-pushing, rebasing, you name it) I'm not sure how you link the current branch to a "changeset" filename.
arcanis
commented
3 years ago Looks like generated filenames are based on git commits but given how people can do a lot of things with their living branches (amending, force-pushing, rebasing, you name it) I'm not sure how you link the current branch to a "changeset" filename.
The names are random hashes, not tied to specific commits. By comparing local branch (feat) versus target branch (master) we know all the files that have been touched between the two. The "current version definition file" is simply the only file from this list that's in the .yarn/versions directory.
akphi
commented
3 years ago So to summarize,
Yarn's current approach is to keep everything in one file only. Pros:
to get a quick overview of every PR that's going to be released, with no duplicates.yarn version check -i we will open the one file and the user can edit the changeset.changesets's current approach is to create a new changeset file every time yarn changeset is called. Pros:
front-matter format for a changeset file, the markdown content doesn't have to follow any convention or need to contain special comments or so.Besides, the format where there are multiple front-matter chunks in markdown file is not supported by gray-matter, so we probably need to chop that file up and parse each chunk separately.
---
"@yarnpkg/cli": minor
---
Does something new
---
"@yarnpkg/core": minor
---
Some API fixBut how do we use yarn version check -i with this? We can't really, or at least if we would want to do that, we need to add logic for the interactive tools: we need to detect there are multiple changeset entries (regardless whether same files or different files) in this PR and ask the user to choose which one they would like to work with.
At this point, and based on my experience with most interactive tools, the editing UX/DX experience offered is not great (maybe because I'm not good at Vim š). The best experience is to bump up my IDE/Text editor and have proper syntax highlighting so I can write some Markdown text. With the interactive tool, I wouldn't be comfortable writing anything more than a one-liner. For example, if I have to inform users about a breaking change and migration path, it wouldn't be one line for sure.
Now I respect both lines of thoughts, but if I really have to choose, I would go with having multiple files. But doing this does not mean to opt-out from Yarn current behavior. In fact, we can detect all the changelog files in the current PR and ask user to choose what they would like to edit, or to add a new one. For this, we can show something like this
> We detected existing changelog entries, would you like to make an edition? (y/N)
# If there is no existing one, we will jump to create new, if they user say No here, we will also jump to create new
# If they choose to `create new` we use the existing `interactive` mode of `yarn version check`
# If they choose to edit, we will show them the following
> Select which changelog entry you would like to make an edit
o a89asd: This changelog is about ....
o 4123a0: This changelog is about ....
o 90adl2: This changelog is about ....
o 13jadd: This changelog is about ....
0 <changelog_name>: <changelog_summary>So there are 2 things for each line:
<changelog_summary>, we can do something like minifying the markdown content of the file (replacing \n with a space and a period or something like that)<changelog_name>, now we can use a random hash. Or we can ask the user to put down something meaningful when they create a new changelog entry like 'fix timeout issue' (of course this question in the interactive mode should be optional as well, if they skip it, we use a hash by default). If people give their changelog a name, we can store the proper name in the front-matter of the changelog file and display it nicely when they are in the menu to select changelog to edit.# If they choose to edit, we will show them the following
> Select which changelog entry you would like to make an edit
o fix timeout issue: lorem ipsum ...
o add mock handler: lorem ipsum ...
o fix window style: lorem ipsum ...
o 13jadd: lorem ipsum ...If we want to simulate the feeling of having 1 file per PR, we can also use a directory. If we have multiple changelog entries, auto-promote changelog to a directory. After all, during yarn version apply we can always flat down everything
.yarn/version
|__ y1467a.yml
|__ olkd23
|__ olp234.yml
|__ 89akup.ymlSo in the top-level view of the .yarn/version directory, we will see that effect of having one file per PR, or heck it, always start with a folder.
.yarn/version
|__ y1467a
|__ plp234.yml
|__ olkd23
|__ olp234.yml
|__ 89akup.ymlHaving this one file per PR rule hasn't been a problem since the introduction of the plugin, and we dogfood it everyday. I don't think we should fix what isn't broken.
If later on we notice problems then we can think about it, but for now we should focus on changelogs and how to integrate them in the current workflow (rather than replace it).
akphi
commented
3 years ago @arcanis sure thing! I just pushed the conversation a bit more so we know roughly what future awaits š I will find some time to work on this given I just spent a fair amount of time working with both tools, my knowledge is still fresh here.
arcanis
commented
3 years ago Awesome, thanks! Feel free to ask on our Discord if you have questions! I myself will be afk a lot this weekend, but other contributors will be around to help you get started š
Andarist
commented
3 years ago The names are random hashes, not tied to specific commits. By comparing local branch (feat) versus target branch (master) we know all the files that have been touched between the two. The "current version definition file" is simply the only file from this list that's in the .yarn/versions directory.
Oh, gotcha - we do similar stuff to filter packages that have been touched. Given that it should be pretty straightforward to implement a UI that would handle either editing the existing changeset or adding a new one.
I see how you might want to have a simple list that would map 1 to 1 to PRs but ditching that seems like a little thing to sacrifice for a simpler mental model around what is allowed and what is not in changeset files. You could also share the same hash for generated changesets (just with some prefix) that would "group" them on the list as related changesets would always be sorted together.
Besides all of that - you probably should implement handling of multiple changeset files anyway. Those files can be easily created manually - a CLI is just a convenience. I assume you would handle such scenario gracefully, without crashing or forcing the user to "fixup" the files manually.
Having this one file per PR rule hasn't been a problem since the introduction of the plugin, and we dogfood it everyday. I don't think we should fix what isn't broken.
Sure it works - same for the other approach. It's not really a discussion about what is workable but rather about tradeoffs. It's IMHO easier to flesh this out now and just not revisit it in the future. I will, of course, respect if you want to keep a single file per PR approach but I truly believe this is mostly an aesthetics issue which also has some little drawbacks. It's sort of like a Prettier situation - you might not agree with all its formatting choices, but at the end of the day you accept that and you don't really look back.
jinlinux
commented
2 years ago Anyone still working on this? Or even make yarn version work more like standard-version would be awesome.
akphi
commented
2 years ago @jinlinux Unfortunately, I lost motivation to work on #2358 as we decided to go with https://github.com/changesets/changesets and I didn't really have time to close this loose end afterwards. Feel free to pick up where I left off in #2358. Sorry about that.
borekb
commented
2 years ago Just yesterday, we were also discussing how to maintain our changelog and I remembered about this issue. So it's funny that you should bring this up, @jinlinux š.
Changesets look like our next best option but for some reason, I'd prefer if this was integrated into Yarn. But I understand that it hasn't been a high-enough priority for anyone yet.
SimenB
commented
2 years ago @SimenB I learned a lot from the way you guys work on facebook/jest
Could you kindly share with me what exactly is preventing you from removing the
lernaat this point? I doubt it's thischangelogfeature that you are waiting for right? - because as I can see, for each PR, you manually add toCHANGELOG.mdfile. So is it something about the release/publish process that makes you stick withlerna?
Seems I never responded, sorry about that! My main objective is to get away from Lerna to reduce the number of tools/dependencies. But currently the version plugin just says what version a package shall get, it doesn't prompt for a changelog entry. What I want is probably changesets (but I don't want another tool - yarn version is so close). I also want to keep locked versioning (I don't want a GH release for every single package, I prefer one entry per shared release. I've stopped watching release of multiple repos using changesets as whenever they made a release my GH notifications got useless).
So my ideal workflow (I think, take with a pinch of salt as this is from memory and 20 seconds of googling for docs on my assumptions)
CHANGELOG.md to avoid merge conflicts) (also needs ignore patterns). Asking people manually to update the changelog is a chore I'd like to get away from (could set up a GH action for that I suppose, but non-code doesn't need it (but sometimes useful), so easier if a tool already understands what's code and what's not and go from there).yarn/versions or otherwise) and create a new combined changelog entry in CHANGELOG.mdMain thing annoying me about our current Lerna based workflow (aside from being a separate tool from the package manager) is that it doesn't support workspace: protocol leading to commits updating the lockfile after a publish (and way bigger diff than needed). A manual changelog is a small price to pay when the other tools don't solve the problem the way I want š
What's stopping me from using yarn version:
dist-tag (at least not mentioned in https://yarnpkg.com/features/release-workflow, I see I can pass pre as strategy from yarn version --help, but no info on what dist-tag is picked)What's stopping me from using changesets:
(and very specific to Jest is that I don't have access to repo secrets (and getting FB/Meta to grant that access is impossible, also for employees there - let alone externals), so using a GH action to publish is currently impossible, meaning that advantage of changesets falls away entirely)
arcanis
commented
2 years ago Every PR gets asked to provide a checked in "changelog" entry (a separate file from the shared
CHANGELOG.mdto avoid merge conflicts) (also needs ignore patterns). Asking people manually to update the changelog is a chore I'd like to get away from (could set up a GH action for that I suppose, but non-code doesn't need it (but sometimes useful), so easier if a tool already understands what's code and what's not and go from there)
One thing I wondered was how much changelog entries should be split. For example, in theory we could make changes in @yarnpkg/fslib that we would add in its changelog as "Adds new whateverSync utility function" but wouldn't be listed in the @yarnpkg/cli changelog because not relevant to end users. Or do you imagine a single changelog for the "main" package, and that's it?
No way of setting
dist-tag(at least not mentioned in https://yarnpkg.com/features/release-workflow, I see I can passpreas strategy fromyarn version --help, but no info on what dist-tag is picked)
At the moment we always set latest unless yarn npm publish requests something else, but I intend to change that to push to canary (or some other tag) if the published version has a prerelease specifier. And stable releases would be pushed to both latest and canary.
SimenB
commented
2 years ago Every PR gets asked to provide a checked in "changelog" entry (a separate file from the shared
CHANGELOG.mdto avoid merge conflicts) (also needs ignore patterns). Asking people manually to update the changelog is a chore I'd like to get away from (could set up a GH action for that I suppose, but non-code doesn't need it (but sometimes useful), so easier if a tool already understands what's code and what's not and go from there)One thing I wondered was how much changelog entries should be split. For example, in theory we could make changes in
@yarnpkg/fslibthat we would add in its changelog as "Adds new whateverSync utility function" but wouldn't be listed in the@yarnpkg/clichangelog because not relevant to end users.
In --deferred I'd then want only @yarnpkg/fslib to get an entry (I guess the interactive prompt would ask me and I'd say "reject" or something for @yarnpkg/cli).
Or do you imagine a single changelog for the "main" package, and that's it?
I'd want a single combined one saying "[@yarnpkg/fslib]: Adds new whateverSync utility function" or some such, but no entry for @yarnpkg/cli.
Note that my preferences might not be transferable to an automatic tool. I might e.g. want a * package change when it touches all packages (such as dropping a node version) and not an entry for every single package. That's a per change/PR sorta thing tho, so maybe š
No way of setting
dist-tag(at least not mentioned in yarnpkg.com/features/release-workflow, I see I can passpreas strategy fromyarn version --help, but no info on what dist-tag is picked)At the moment we always set
latestunlessyarn npm publishrequests something else, but I intend to change that to push tocanary(or some other tag) if the published version has a prerelease specifier. And stable releases would be pushed to bothlatestandcanary.
Ah, wonderful!
I'd personally use canary as a per-commit (or possibly nightly) release thing, and keep next for e.g. alpha releases. But as long as the tag is configurable that sounds great š
Semi-related, what happens to the files in .yarn/versions? Will yarn still be able to look for changes since last stable release when the time comes to promote?
arcanis
commented
2 years ago Semi-related, what happens to the files in .yarn/versions? Will yarn still be able to look for changes since last stable release when the time comes to promote?
Yep. The drawback however is that during prereleases, all packages are deployed each time we make a prerelease (Yarn doesn't know which ones have been deployed before, since the files are still there). Could be improved, perhaps.
SimenB
commented
2 years ago Could look at git tags? Regardless, not the biggest blocker š
Describe the user story
Yarn has the awesome deferred version strategy built-in into version plugin, but currently, it doesn't support changelogs. I want to understand what changes were introduced in specific version of package.
Describe the solution you'd like
It could be part of version plugin or separate plugin too. I expect to work it in a similar way as changesets. So a user will add changelog entry when bumping version. And later, when we apply versions, we should generate a markdown file with these entries.
Describe the drawbacks of your solution
If it will be part of the version plugin, we need some options to enable/disable changelogs because not everyone needs it. Is it doable at all without breaking change?
Describe alternatives you've considered
Btw there is the issue about yarn integration with changesets. Maybe we can enhance changesets a bit so it will work like standalone cli.
Additional context
Maybe there is some other tools (like lerna)?