search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.16k stars 3.82k forks source link

jobs: remove batching from job scheduler #78661

Open nvanbenschoten opened 2 years ago

nvanbenschoten commented 2 years ago

The job scheduler currently attempts to schedule up to 10 jobs in the same transaction. This is unnecessary from a throughput perspective, as job scheduling throughput is very low. Meanwhile, it increases the size of transactions, increasing the change of transaction retries. It also makes the job scheduler queries more complex and potentially susceptible to suboptimal query plans. Unless there's a strong reason to keep this batching, we should remove it.

cc. @miretskiy

Jira issue: CRDB-14206

miretskiy commented 2 years ago

I'm a bit hesitant to remove batching entirely... We can do that by changing cluster setting. But in this PR https://github.com/cockroachdb/cockroach/pull/78564, perhaps we can keep batching w/out all of the negatives we observed.

miretskiy commented 1 year ago

Moving back to triage; I don't think we should do anything on this front right now. Feel free to close.