cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.1k stars 3.81k forks source link

cdc: protect all required tables under pts #128806

Open wenyihu6 opened 2 months ago

wenyihu6 commented 2 months ago

Describe the problem

For standard changefeeds, CDC only protects the DescriptorTableID table when creating the PTS record. https://github.com/cockroachdb/cockroach/blob/8c7a79c0f1b66578a03d825bf63a89cda66d84bc/pkg/ccl/changefeedccl/protected_timestamps.go#L54 However, this is insufficient for CDC, since other system tables are required to read descriptors. For CDC queries there are additional requirements, as the query planner requires access to additional tables for checks. We should at least identify which tables are needed to resume changefeeds and ensure they are properly protected. A more robust, less error-prone solution would be more ideal though.

More details in https://github.com/cockroachlabs/support/issues/3008.

Tables that have been identified as required so far:

Jira issue: CRDB-41217

Epic CRDB-41784

blathers-crl[bot] commented 2 months ago

Hi @wenyihu6, please add branch-* labels to identify which branch(es) this C-bug affects.

:owl: Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

blathers-crl[bot] commented 2 months ago

cc @cockroachdb/cdc

rharding6373 commented 1 month ago

I've seen some evidence in cloud that the system.role_members may be a dependency for enterprise changefeeds without cdc queries as well.