search

procrastinate-org / procrastinate

PostgreSQL-based Task Queue for Python
https://procrastinate.readthedocs.io/
MIT License
840 stars 52 forks source link

3.x ? #1135

Open ewjoachim opened 1 month ago

ewjoachim commented 1 month ago

Relates to #1131 and #1114 (and a few others)

I was thinking about whether we were doing breaking changes in #1114 and you mentionned it in #1131 so maybe it's break time.

Version numbers are free. The downside of doing a major release is that we're using our limited voice power to call the attention of every downstream user of the lib, and if we do it too often, there's fatigue and "boy who cried wolf" effect. We already did 2 major releases (although in quick succession) at the beginning of the year, I think it's perfectly OK to do a new one especially since we're adding plenty of good stuff to the lib (thank you both !), but it might be worth it to try and think about the decisions we did that we ended up changing, and see if we identify a pattern in what led to that so as not to find ourselves in the same situation again too soon. To be clear: I'm not criticizing any decision we made, but trying to get as many opportunities as we can to improve ourselves, our vision for the lib, the processes etc.

If we want to drop sync_to_async at the same time, it might be an opportunity.

Things to note:

(also, each time we do a major release, it seems we're featured in Postgres Weekly, but I don't think they'll do it if we release too often so let's not abuse this :D )

ping @onlyann @medihack

medihack commented 1 month ago

First, let us collect those relevant issues to provide an overview. I first created a label, but a milestone is more appropriate. I will call the milestone "Version 3.0".

medihack commented 1 month ago
  • I think it's about the first time where we have a few unrelated things to add to a single release. Do we want to create a milestone to track it ?

I already added 3 issues to the created milestone. Anything else?

  • Do we need to advertise a "provisional API" period when we introduce new features? this might help with such cases in the future and let us break young features more easily.

I think it depends on how many features we plan to add. From looking at other distributed task libraries, there is not much missing (if at all), but I could be totally wrong 😉 (I'm not sure what your plans are). Every new administrative concept also comes with an additional burden.

  • In this major version, we added DeprecationWarnings for the first time. What do we want to do with those in 3.x? Also, should we try and make this a habit more ?

Yes, where it makes sense. If we replace something with something better, but leave the old thing there.

  • We can do a branch for 3.x or we can develop 3.x on main. Depends whether we think it's going to be long and whether we think we might want to do things in parallel.
  • We can do beta versions if it helps. We can delay releasing the real 3.x until we're satisfied with the beta.

If we decide to do a beta version then yes, I think it's worth having a separate branch. There are currently only 3 issues, and it seems to me that we can tackle them quite fast. So, I am not sure if it is worth it. On the other hand, the worker refactoring seems to be quite large, and having a beta version would give us some safety (@onlyann, what do you think?).

  • A major version means we get to clean our schema a little. In 3.0 we keep the schema, but after 3.0 we can immediately issue a 3.1 where we drop all the old versions of the SQL procedures. See the doc.

I am absolutely in favor of this. I wanted to clean up the comments there all the time anyway.

(also, each time we do a major release, it seems we're featured in Postgres Weekly, but I don't think they'll do it if we release too often so let's not abuse this :D )

And maybe some posting on Hacker News 😜.

onlyann commented 1 month ago

I fully support the introduction of a beta to obtain feedback and iterate over changes, if that is not too much work to put in place.

Good idea to introduce a tag.

I will take the opportunity to open new issues about some breaking change suggestions.

medihack commented 1 month ago

Should we create a next or v3 branch and add it to the CI action?

ewjoachim commented 1 month ago

Sure! Though I'm not sure we need to change things in the CI (we can add the fact that the CI runs on the branch after we merge), but we can apply the same branch protections rules.

medihack commented 1 month ago

Sure! Though I'm not sure we need to change things in the CI (we can add the fact that the CI runs on the branch after we merge),

Hm. If I understand it correctly, we want all the breaking stuff for the 3.0 release in that branch. So, there will be multiple PRs against this branch. Don't we want to run CI against those PRs before they get merged?

but we can apply the same branch protections rules.

Will you set it up? I don't think that I have the permissions to change the protection rules.

ewjoachim commented 1 month ago

Don't we want to run CI against those PRs before they get merged?

Unless I'm mistaken, CI runs on all PRs regardless of the target branch AND on main to double-check that what we merged didn't cause an issue. If we target v3, it will run on PRs all the same, it's just that it won't run on the v3 branch unless we request it so.

Will you set it up? I don't think that I have the permissions to change the protection rules.

Yep.

ewjoachim commented 1 month ago

Here you are. There's a v3 branch, and I've set up the same branch protection rules on v3 as on main. CI will run on PRs that target v3 (that was already the case before). If we want to run the CI on v3 itself we can either set up the ci to do so, or just do a PR from v3 to main.