temporalio / temporal

Temporal service
https://docs.temporal.io
MIT License
10.3k stars 754 forks source link

SQL Support for Backlog Counter #5915

Closed Shivs11 closed 1 week ago

Shivs11 commented 2 weeks ago

What changed?

Support for the backlog counter, which was added in #5593, results in Cassandra persisting the backlog counts when tasks are created. However, the same is not being done by SQL databases as we don't persist task queue related information while creating tasks. Thus, if we fetch information immediately after we have created tasks, the backlog counter for SQL databases would read 0 even when we do have tasks in our backlog.

To fix this, a Count(*) query has been implemented whenever we fetch task queues.

Why?

This is required for the backlog counter accurate for both Postgres and SQL databases.

How did you test it?

Potential risks

None, since the backlog counter work has not been released yet.

Is hotfix candidate?

No

Shivs11 commented 2 weeks ago

Note : Discussions with @dnr and @ShahabT reached a consensus which requires a config to be added in order to allow customers to trigger the option of calling Count(*) or not. This shall be added *after this branch merges into shivam/backlog-count-updated and once this branch is looking to be merging into main. This is because of the fact that the current branch is not up-to-date with main, which has undergone config-changes recently. Thus, to avoid repetitive work, this shall be done when merging the feature into main.

EDIT - Since this will not be merged to main, the config will not be added.