cockroachdb / cockroach

CockroachDB - the open source, cloud-native distributed SQL database.
https://www.cockroachlabs.com
Other
29.52k stars 3.7k forks source link

rangefeed: client-controlled closed timestamp interval #114134

Open erikgrinaker opened 8 months ago

erikgrinaker commented 8 months ago

In large clusters, closed timestamp processing accounts for a large fraction of changefeed resource usage. Often, the changefeed will run with a large checkpoint interval (default min_checkpoint_frequency = 30s), but the default rangefeed closed timestamp interval is much smaller (kv.rangefeed.closed_timestamp_refresh_interval = 3s), which can produce unnecessary overhead.

It would be convenient if the rangefeed client could request a specific refresh interval, to avoid sending these more often than needed.

This would also be useful for system rangefeeds, that are low-volume but may also want low latency (see e.g. span config updates which are only applied after a checkpoint).

Jira issue: CRDB-33364

blathers-crl[bot] commented 8 months ago

cc @cockroachdb/replication