mrpowers-io / quinn

pyspark methods to enhance developer productivity 📣 👯 🎉
https://mrpowers-io.github.io/quinn/
Apache License 2.0
647 stars 99 forks source link

Create the automated release process using googleapis/release-please #262

Open nijanthanvijayakumar opened 2 months ago

nijanthanvijayakumar commented 2 months ago

Proposed changes

Types of changes

What types of changes does your code introduce to quinn? Put an x in the boxes that apply

Further comments

dinjazelena commented 2 months ago

@nijanthanvijayakumar @SemyonSinchenko . It should be on on_push. So basically what happens is when u want to introduce the change to your main branch, most likely from PR or just a single commit. Every time there is commit(or PR), release-please-action is triggered and it searches for pattern in your commit message(PR title) with prefix of fix: or feat: or fix! or refactor! or ci: and many more. Based on your PR title(commit message) release please would create a PR which is using to accumulate and follow all of your changes.

So lets say u have window for releasing, and by that time u made 3 bug fixes added 2 new features, release please keeps all of those changes in your automatic pull request. Once u are ready to merge that automatic pull request created by release please, it would create a Github Release, bump versions in __init__.py, pyproject.toml , update CHANGELOG.md and ideally release your artifact to PyPi.

What release please lacks is using both prelease and release, it promotes trunk based, but it somehow does not allow it to use both of these. So u can automatically create for example release candidates, and then just have a manual workflow to release production version or vice versa.

And ideally u tested this on separate github repo to make sure its all good.

Also release-please-manifest.json, is what is used to track versioning under the hood, so current latest version should be there.

SemyonSinchenko commented 2 months ago

@nijanthanvijayakumar @dinjazelena Thanks for the explanation! Do I understand right, that this workflow make a realease to PyPI? If so, which secrets should be provided?

nijanthanvijayakumar commented 2 months ago

@SemyonSinchenko - this workflow creates a GitHub release as in the screenshot below and I don't think it creates a PyPi release (I might take your help in double-confirming this @dinjazelena?). The only reference to Python I used in this workflow is here.

While creating it, I didn't see any references to PyPi and it worked on the test repo I shared earlier without involving the creation of any secrets. image

SemyonSinchenko commented 2 months ago

Can we somehow add a step to the release pipeline that publish in PyPI release or pre-release versions?

nijanthanvijayakumar commented 2 months ago

Can we somehow add a step to the release pipeline that publish in PyPI release or pre-release versions?

Sure can look into it, but did you want that to be done in this PR?

dinjazelena commented 2 months ago

Yeah, so this only creates Github Release, but logically, u would just have additional step in your github action workflow with smth like this

if: ${{ steps.release.outputs.release_created }}
run: |
   poetry publish ...

Here is an example for nmp.

Question is only do you want to create automatic release for pre-releases or for production releases because unfortunately it cannot support both.

SemyonSinchenko commented 2 months ago

Question is only do you want to create automatic release for pre-releases or for production releases

I would have better automated prod-release

nijanthanvijayakumar commented 2 months ago

Question is only do you want to create automatic release for pre-releases or for production releases

I would have better automated prod-release

@SemyonSinchenko - sure. I came across this article, and according to this, one doesn't need secrets to publish to the PyPI repository due to the "trusted publishing" feature. However, as indicated in the image below from the Set up PyPI publishing using GitHub section of that article, Quinn's project author or maintainer must add GitHub Actions as a trusted publisher for that project. Are you or @MrPowers able to do that while I whip up the code for publishing the package to PyPI?

Steps for adding a trusted publisher is documented in this PyPI docs here - https://docs.pypi.org/trusted-publishers/adding-a-publisher/

image

nijanthanvijayakumar commented 2 months ago

Question is only do you want to create automatic release for pre-releases or for production releases

I would have better automated prod-release

@SemyonSinchenko - sure. I came across this article, and according to this, one doesn't need secrets to publish to the PyPI repository due to the "trusted publishing" feature. However, as indicated in the image below from the Set up PyPI publishing using GitHub section of that article, Quinn's project author or maintainer must add GitHub Actions as a trusted publisher for that project. Are you or @MrPowers able to do that while I whip up the code for publishing the package to PyPI?

Steps for adding a trusted publisher is documented in this PyPI docs here - https://docs.pypi.org/trusted-publishers/adding-a-publisher/

image

Hello @SemyonSinchenko ! Any suggestions on the next steps?

SemyonSinchenko commented 2 months ago

Sorry, I missed the comment. I will ask @MrPowers

nijanthanvijayakumar commented 2 months ago

Sorry, I missed the comment. I will ask @MrPowers

Thanks @SemyonSinchenko

SemyonSinchenko commented 2 months ago

@nijanthanvijayakumar Do I understand you right that all the work is done and the only thing left is adding a trusted publihser to PyPI?

nijanthanvijayakumar commented 2 months ago

Yep, @SemyonSinchenko - here's the commit that contains the changes that help publishing the code to the PyPI.

SemyonSinchenko commented 2 months ago

@nijanthanvijayakumar LGTM overall, thanks for the effort and especially for the patience! @dinjazelena Hi! Can I ask you to take a final look on it too?

dinjazelena commented 2 months ago

But maybe u should guys use much more simpler way, i know i recommended this release please. What do u think about this approach. release-snapshot -> On every push to main snapshot version is created for testing release -> On every pushed tag production version is created

And all of that can be done via: https://github.com/softprops/action-gh-release

It requires few clicks more, like taking care of version number and tag. But it feels like simplicity overweights that especially with how release-please can be very hard to setup and understand.

zeotuan commented 1 month ago

But maybe u should guys use much more simpler way, i know i recommended this release please. What do u think about this approach. release-snapshot -> On every push to main snapshot version is created for testing release -> On every pushed tag production version is created

And all of that can be done via: https://github.com/softprops/action-gh-release

It requires few clicks more, like taking care of version number and tag. But it feels like simplicity overweights that especially with how release-please can be very hard to setup and understand.

@SemyonSinchenko @MrPowers I actually Prefer this method mentioned above which I sometimes call semi-automatic release over fully automatic prod release. This way you can control when an actual release is supposed to be published and avoid accidental release of non-stable feature.

It's simpler, safer and easier to manage. Whoever want to use the nonstable version of the product can always pull the snapshot