search

procrastinate-org / procrastinate

PostgreSQL-based Task Queue for Python
https://procrastinate.readthedocs.io/
MIT License
873 stars 55 forks source link

3.x ? #1135

Open ewjoachim opened 4 months ago

ewjoachim commented 4 months ago

Relates to #1131 and #1114 (and a few others)

I was thinking about whether we were doing breaking changes in #1114 and you mentionned it in #1131 so maybe it's break time.

Version numbers are free. The downside of doing a major release is that we're using our limited voice power to call the attention of every downstream user of the lib, and if we do it too often, there's fatigue and "boy who cried wolf" effect. We already did 2 major releases (although in quick succession) at the beginning of the year, I think it's perfectly OK to do a new one especially since we're adding plenty of good stuff to the lib (thank you both !), but it might be worth it to try and think about the decisions we did that we ended up changing, and see if we identify a pattern in what led to that so as not to find ourselves in the same situation again too soon. To be clear: I'm not criticizing any decision we made, but trying to get as many opportunities as we can to improve ourselves, our vision for the lib, the processes etc.

If we want to drop sync_to_async at the same time, it might be an opportunity.

Things to note:

(also, each time we do a major release, it seems we're featured in Postgres Weekly, but I don't think they'll do it if we release too often so let's not abuse this :D )

ping @onlyann @medihack

medihack commented 4 months ago

First, let us collect those relevant issues to provide an overview. I first created a label, but a milestone is more appropriate. I will call the milestone "Version 3.0".

medihack commented 4 months ago
  • I think it's about the first time where we have a few unrelated things to add to a single release. Do we want to create a milestone to track it ?

I already added 3 issues to the created milestone. Anything else?

  • Do we need to advertise a "provisional API" period when we introduce new features? this might help with such cases in the future and let us break young features more easily.

I think it depends on how many features we plan to add. From looking at other distributed task libraries, there is not much missing (if at all), but I could be totally wrong 😉 (I'm not sure what your plans are). Every new administrative concept also comes with an additional burden.

  • In this major version, we added DeprecationWarnings for the first time. What do we want to do with those in 3.x? Also, should we try and make this a habit more ?

Yes, where it makes sense. If we replace something with something better, but leave the old thing there.

  • We can do a branch for 3.x or we can develop 3.x on main. Depends whether we think it's going to be long and whether we think we might want to do things in parallel.
  • We can do beta versions if it helps. We can delay releasing the real 3.x until we're satisfied with the beta.

If we decide to do a beta version then yes, I think it's worth having a separate branch. There are currently only 3 issues, and it seems to me that we can tackle them quite fast. So, I am not sure if it is worth it. On the other hand, the worker refactoring seems to be quite large, and having a beta version would give us some safety (@onlyann, what do you think?).

  • A major version means we get to clean our schema a little. In 3.0 we keep the schema, but after 3.0 we can immediately issue a 3.1 where we drop all the old versions of the SQL procedures. See the doc.

I am absolutely in favor of this. I wanted to clean up the comments there all the time anyway.

(also, each time we do a major release, it seems we're featured in Postgres Weekly, but I don't think they'll do it if we release too often so let's not abuse this :D )

And maybe some posting on Hacker News 😜.

onlyann commented 4 months ago

I fully support the introduction of a beta to obtain feedback and iterate over changes, if that is not too much work to put in place.

Good idea to introduce a tag.

I will take the opportunity to open new issues about some breaking change suggestions.

medihack commented 4 months ago

Should we create a next or v3 branch and add it to the CI action?

ewjoachim commented 4 months ago

Sure! Though I'm not sure we need to change things in the CI (we can add the fact that the CI runs on the branch after we merge), but we can apply the same branch protections rules.

medihack commented 4 months ago

Sure! Though I'm not sure we need to change things in the CI (we can add the fact that the CI runs on the branch after we merge),

Hm. If I understand it correctly, we want all the breaking stuff for the 3.0 release in that branch. So, there will be multiple PRs against this branch. Don't we want to run CI against those PRs before they get merged?

but we can apply the same branch protections rules.

Will you set it up? I don't think that I have the permissions to change the protection rules.

ewjoachim commented 4 months ago

Don't we want to run CI against those PRs before they get merged?

Unless I'm mistaken, CI runs on all PRs regardless of the target branch AND on main to double-check that what we merged didn't cause an issue. If we target v3, it will run on PRs all the same, it's just that it won't run on the v3 branch unless we request it so.

Will you set it up? I don't think that I have the permissions to change the protection rules.

Yep.

ewjoachim commented 4 months ago

Here you are. There's a v3 branch, and I've set up the same branch protection rules on v3 as on main. CI will run on PRs that target v3 (that was already the case before). If we want to run the CI on v3 itself we can either set up the ci to do so, or just do a PR from v3 to main.