Semantic Versioning

[JoaoRodrigues]

@joaomcteixeira is right, we should adhere to stricter versioning standards. My suggestion is that each PR bumps the minor version by 1, and once we have major changes, we bump the others. Suggestions on how to do this automatically?

Some ideas:

• https://pypi.org/project/versioneer/
• https://github.com/c4urself/bump2version/#installation

[joaomcteixeira]

I agree to adhere to stricter versioning standards. Though I disagree that each PR should bump the minor version number :stuck_out_tongue_winking_eye: . Each PR should bump whatever version number is appropriate, following Semantic Versioning 2.0, these can be major, minor or patch numbers.

The workflow I follow in my other projects is the following:

1. developer sends PR and proposes a version change: major, minor or patch
2. PR should also address changes in CHANGELOG
3. the PR itself cannot be accompanied by a version change itself because it is not guaranteed that PR will be the next to be merged.
4. if the PR is accepted, the main repository maintainer must:
   1. updated the CHANGELOG accordingly
   2. bump the version accordingly
   3. I normally do this by committing directly to the master.
   4. I accompany version bump with a new git tag.
5. No new PRs should be accepted while the version has not been bumped after the previous PR.

Note that this is my workflow for an agile development/deployment practice. It differs from a periodic delivery. I also do not use a develop branch, it conflicts with the Semantic Versioning principles.

I never attempted to automate the bump version process in the CI workflow, but this is something interesting to do. Also, integrating the deployment of the new version on PyPI in this chain of actions, especially in the CI, is also a good practice.

[mtrellet]

That's a discussion that comes right on time!

I've recently created a package used in one of our solution in development (so internal usage only for the moment). I created a CI pipeline rather early in the process and I had to ask myself how I'll handle the versioning to keep things clean.

I do like your idea of having the maintainer in charge of the version bump following the dev proposition. As you perfectly stated it happens quite often than a MR/PR is not merged in the order it has been created.

Here we also have an extra "difficulty" since the package is sent, within the CI pipeline, to a package manager online (like a private PyPI). And we must make sure that the next version is 1) following nicely the previous one, 2) reflecting the amount of changes it brings (major, minor, patch). I have a script that checks the last version present on the server and prevents the package to be pushed there is the version has not changed. Allows me to avoid an error reported when trying to push the same version of a package. But this doesn't solve at all the issue with automatically bumping the version without creating a specific commit (be it merged with the previous commit or not...).

In summary, I won't bring much to the discussion, just supporting the pipeline you've proposed! And I'll submit it to my colleagues to brainstorm around it ;)

[JoaoRodrigues]

Good discussion! My idea was to integrate one of the tools with the CI. I might also consider moving to Azure Pipelines, since I have it setup for another project and I like it better than Travis CI. There we can have a step where the CI checks if any new commit to the master branch was successful and if so, increase the version number automatically. I'm assuming we'd want finer control of our major upgrades so these would be not automated.

e.g. https://stackoverflow.com/questions/42718373/increase-version-number-if-travis-at-github-was-successful

[joaomcteixeira]

Hi,

1. Thanks for the bump2version! :wink: I am using the original bumpversion and I had missed that new and improved fork. :cool:
   1. I never used versioneer. I have always used bumpversion and I really like the way it works, but no restrictions on using versioneer.
2. I agree with the strategy on the stackoverflow post. I agree that, for each commit to the master, the CI integration bumps the version, commits to github with tag, and deploys in PyPI. This configuration assumes there are no direct commits to the master, additions come only via PRs.
3. I am okay using AzurePipelines @JoaoRodrigues :stuck_out_tongue_closed_eyes:

EDIT: GitHub Actions has a special action for PyPI package upload. I am testing it now for other repositories. - for pdb-tools.

[joaomcteixeira]

Just a small addition. I have been using the GitHub Actions to deploy taurenmd to PyPI automatically every time a new release is created in the repository and so far is working very well. :+1:

I am starting to consider if it is worth migrating the whole Travis-CI to GitHub actions. :eyeglasses:

[JoaoRodrigues]

Trying to get this up and running this morning. The idea is to minimize the work going from accepted pull request to published package on PYPI. I'll be editing this comment as I work.

Here's a first attempt. We can use two Github actions in tandem, one that makes a new release from every accepted pull request, and another that published the package on PYPI on every new release. This is super simple to setup but has a few issues, namely, how do we define version numbers? How do we know which version number a pull request should change: major, minor, or patch?

[joaomcteixeira]

• everytime a PR is accepted, CI runs again on the new commit to the master branch
• we can configure CI such that when running on master and, if passes, bumps the version and commits back to master, and then publish a release.
  • we need a flag here to differentiate from merging PRs and PRs from the CI itself after version bump.
• we could parse the commit message (from the PR merge) so to direct the version bump. (bumpversion: patch) for example.
• Git Actions would do the final trick.

[JoaoRodrigues]

That's where am I at now. I settled on bump2version, every push to master triggers a new version bump depending on the commit message of the merge commit (so the person that does the merge has to be responsible about this!).

[JoaoRodrigues]

Here's the workflow and the progress:

• [x] Trigger bump2version on merge commit
• [x] Differentiate between patch/minor versions through merge commit message.
• [x] Trigger release on new version
• [x] Trigger packaging on new release

[joaomcteixeira]

you can trigger a tag automatically with bumpversion, actually that is the default behaviour if I am not wrong.

[JoaoRodrigues]

That will be helpful for the release part. Thanks.

[JoaoRodrigues]

Argh. Finally! I went with a simple route in the end because Github Actions are quite restrictive and you cannot easily trigger jobs from each other. Also tried playing with personal access tokens but failed. So, simple solution.

There is one job that checks if the last commit message has [SKIP] in it. If it does not, then the job runs. This is useful if we want to push documentation changes that should not trigger new releases. If the commit message has [FEATURE], it updates the minor version number. Otherwise, it updates the patch number. Major version changes should be handled manually.

I've tested the packaging/bumping action on my own fork, and it should work. I'll make a pull request now and integrate it, and will test it afterwards with an actual bug fix!

[JoaoRodrigues]

Tests have passed! Checking now a real test to push to the production PyPi. This will bump the patch number by a few, but it's OK.

[JoaoRodrigues]

Works! Version 2.0.3 deployed automatically on PyPi. I also use tags to make it show in our releases page.

[mtrellet]

(Previous comment in PR discussion, I figure out that it would make more sense here)

Congrats! Seems that it was quite challenging if I look at the list of commits..! But the results and the new pipeline looks really nice!

Just for the future, you certainly know it but to avoid to have a commit history too long you can try an interactive rebase of your local history before merging it to the main repo. This allows to delete, merge or re-order commits. It's relevant if some commits are not really interesting and helpful to understand the dev process. Or more brutal, if this makes sense, squash the commits upon merging!

Quick question: Does the .bumpversion.cfg needs to be updated as well when committing? This could be added to the README as well if this is the case.

[JoaoRodrigues]

Seems that it was quite challenging if I look at the list of commits..!

Painful...

Just for the future, you certainly know it but to avoid to have a commit history too long you can try an interactive rebase of your local history before merging it to the main repo. This allows to delete, merge or re-order commits. It's relevant if some commits are not really interesting and helpful to understand the dev process.

I know, I did rebase!! I just left the ones I thought were important in case we want to revert some part of the process at some point. I could have also squashed, but again, same reason.

Quick question: Does the .bumpversion.cfg needs to be updated as well when committing? This could be added to the README as well if this is the case.

Nope. It's all automatically updated. Running bump2version will update setup.py and bumpversion.cfg.