ci: improve release workflow

[DaniBodor]

An automated workflow can be triggered from the Actions tab (once this is merged to main) that creates a draft GitHub release. The draft release (notes) can be edited if needed, and then manually converted to an actual Release here, which will trigger the PyPi release (note that a draft release does not trigger PyPi release).

The workflow has the following logic:

• Take inputs:
  • version update level ("patch", "minor", or "major")
  • release branch
  • Unfortunately, the default branch cannot be anything else than main, which is almost never the branch we want to release from. Therefore, if main is selected, the workflow fails, because it was likely by accident forgotten to change the default. Releasing from main can still be done manually if needed.
• Merge release-branch into main
  • A conflict will terminate the process before release
• Bump the version
  • install bump-my-version, bump the version on main, push to remote.
  • NOTE: bump2version is no longer maintained, so its maintained fork bump-my-version is used instead.
• Create draft release
  • the draft can then manually be converted into a published release.
• If above is successful, a separate job is started to clean the workspace:
  • In this way, the "release" will not be listed as failed if below doesn't work (usually due to conflict with dev branch)
  • changes are merged into dev
  • the associated PR is closed
  • if released from a different branch than dev, the hotfix branch is deleted.

This workflow was adapted from here.

fix #170

[DaniBodor]

Very nice improvements to the workflow :D

I added some comments, and requested changes since I'd like to be aware of them (the flow is a bit more complex now, I am like a new developer testing it). Also I wonder, has this workflow being tested? Maybe we can try to do a draft release (that then we cancel) before merging it into dev.

I've made some improvements, mainly to the documentation.

I've tested this in the eitprocessing repo, but not yet here. There are a few difficulties in testing this:

• Each time the workflow is triggered, changes are merged into main (even if the release is just a draft). To avoid this, I had created test branches in the other repo and pretended those were the main to keep the true main clean.
• Until the workflow is not on main, it cannot be triggered from the actions tab. This can be somewhat circumvented by using GitHub CLI with this command gh workflow run "Draft GitHub Release" --ref 170_release_workflow_dbodor -f version_level="patch", but it will always read main as the release branch and fail.
• both issues above mean temporarily changing the workflow while testing, which is time consuming.

EDIT: newest tests on eitprocessing aren't working after all 💩 . I will figure it out there and then make changes here. Until then, this PR can stay open.

[gcroci2]

EDIT: newest tests on eitprocessing aren't working after all 💩 . I will figure it out there and then make changes here. Until then, this PR can stay open.

Okay, I've added a couple of comments still to be addressed. Just ask for my review again when you'll discover what is the fix needed :)

[DaniBodor]

EDIT: newest tests on eitprocessing aren't working after all 💩 . I will figure it out there and then make changes here. Until then, this PR can stay open.

Okay, I've added a couple of comments still to be addressed. Just ask for my review again when you'll discover what is the fix needed :)

Will do. I think the best way to test this is to create a fork of the repo, test everything there, and then if it works implement it here as well.

[gcroci2]

Is this ready to be reviewed again? @DaniBodor If not, maybe either remove me from the reviewers or make this PR a draft (otherwise I keep receiving notifications ;))

[github-actions[bot]]

This PR is stale because it has been open for 14 days with no activity.

[DaniBodor]

This action can now be tested on this test fork: https://github.com/EIT-ALIVE/deeprank2_test_release/. I tested it already and it worked. See if you can follow from the dev readme how to do it and if it's explained clearly enough.

Note that in the forked repo, I bugged the release-on-PyPi action, so that we can still check that it gets triggered, but then fails (I don't think it would have been allowed to release a different repo, but didnt want to find out that somehow it does).

Also, I believe all the issues you @gcroci2 flagged have been addressed, or did I miss anything?

[github-actions[bot]]

This PR is stale because it has been open for 14 days with no activity.

[DaniBodor]

The issue is due to the toml file, where the current version of the package seen from bumpversion does not get updated, and thus the error.

Other remaining things before merging:

• Non-expiring token

• Suggestions

It turns out that the error in the versioning is due to another bump version that occurred in between me creating this PR and now, which still used the old system and didn't update the version listed in bump-my-version-settings. Anyway, I've now made the line to be updated in each file explicit (avoids potential isssues e.g. if a dependency by coincidence lists identical version number) and moved the bumpversion settings into a separate toml file to keep pyproject.toml tidier. (NB: I think it would be a good idea to move the ruff settings into a separate toml as well, but that can be done in a separate PR. I create #641 for this.)

See my comments 1/2 on non-expiring tokens

I've integrated all suggestions to the wording of the dev readme, and ensured that the workflow is aborted if dev (or main) branch was selected. Also, I've turned on fail-fast to avoid a situation where bump-my-version (or any other step) fails, but the action is listed as successful (which I think happened in your test runs).

[DaniBodor]

I request changes to remember that we need to look at the security thing again. I added a couple of minor suggestions for the instructions. Also let me know if I have to test everything on the fork again (if you say is all good I'll avoid that)

OK, let me know what you decide. If you want to test again (only minor changes occurred to workflow itself), the please use this new fork instead of the old one. It was quicker to create a new fork rather than update the other one.

• [ ] Once we are done testing, we should delete both test forks.

[gcroci2]

I request changes to remember that we need to look at the security thing again. I added a couple of minor suggestions for the instructions. Also let me know if I have to test everything on the fork again (if you say is all good I'll avoid that)

OK, let me know what you decide. If you want to test again (only minor changes occurred to workflow itself), the please use this new fork instead of the old one. It was quicker to create a new fork rather than update the other one.

• [ ] Once we are done testing, we should delete both test forks.

I tried doing the test, but it gives me: "Releasing from main or dev branch is not permitted, please select a valid release branch." even if I am using a branch called gcroci2-release-test. See here the PR and here the failed action. @DaniBodor

[DaniBodor]

I request changes to remember that we need to look at the security thing again. I added a couple of minor suggestions for the instructions. Also let me know if I have to test everything on the fork again (if you say is all good I'll avoid that)

OK, let me know what you decide. If you want to test again (only minor changes occurred to workflow itself), the please use this new fork instead of the old one. It was quicker to create a new fork rather than update the other one.

• [ ] Once we are done testing, we should delete both test forks.

I tried doing the test, but it gives me: "Releasing from main or dev branch is not permitted, please select a valid release branch." even if I am using a branch called gcroci2-release-test. See here the PR and here the failed action. @DaniBodor

it's fixed