pts: Protected timestamp system has single point of failure

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.

https://www.cockroachlabs.com

Other

29.96k stars 3.79k forks source link

pts: Protected timestamp system has single point of failure #109702

Closed miretskiy closed 7 months ago

miretskiy commented 1 year ago

PTS system uses "meta" table which contains a single row. This table gets updated every time PTS record is updated. If you have many processes that want to manage PTS (e.g. 100s or 1000s of changefeeds) all of these changefeeds will contend against single row update. This does not scale, and causes long running retries/backoffs.

Jira issue: CRDB-31059

gz#18553

Epic CRDB-34798

gz#20330

daniel-crlabs commented 2 weeks ago

Customer is asking if this can be backported to 23.2, they ran into this on 23.1.13 and will be upgrading to 23.2.10

cc @rharding6373 and @jayshrivastava