search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
29.99k stars 3.79k forks source link

protectedts: improve observability of protections that apply to a schema object #82948

Open adityamaru opened 2 years ago

adityamaru commented 2 years ago

Protected timestamp records hold up GC on schema objects during operations such a BACKUP and Changefeeds. A schema object is defined as a table, database, or cluster. It is important to surface information about the records that are holding up GC on the schema object. In particular, what timestamp they are protecting, and who wrote the record. This information is available in the system.protected_timestamp_records table, which is used by the PTS system to persist metadata about PTS records.

We already display the size of a table, database, and cluster in the debug console and this might be an appropriate place from a UX point of view to display what protections apply to the object. Since protections tie in with the Time-To-Live (TTL) of the object, we could also display the TTL determined by the zone config that applies to the object.

The need for this observability has come up a few times in support escalations where the user is left wondering why their data isn't being gc'ed. Either due to orphaned protected timestamp records on dropped tables, or bugs in backup/cdc holding on to protections that are invisible from the debug console.

Jira issue: CRDB-16750

Epic CRDB-32494

amruss commented 2 years ago

Discussion on this topic in this doc: https://docs.google.com/document/d/1m7xqfZ4cYV-qDwE-VU_2b-mc6YiaCt_JZXz4rAn4gsk/edit#

Not sure its a jobs system issue