protectedts: rework protected timestamp storage for multi-tenant model

[adityamaru]

This is a tracking issue for the work that will be required to support protected timestamps in a multi-tenant mode. This is a very high level breakdown of tasks, the understanding of which might evolve as we get to coding them.

RFC https://github.com/cockroachdb/cockroach/pull/74685

Client side:

• [x] #74211: Change ptpb.Record to reflect that we will be protecting objects instead of spans
• [x] #74248: Populate the newly added target field at all call sites were we construct a ptpb.Record. (Backup, CDC, tests)
• [x] #74281 Change the system.protected_timestamp_records table schema to have an additional BYTES column to store this target information. This includes adding a migration for clusters being upgraded.
• [x] #74297 Refactor the Protect method (and other ptpb.Storage interface methods) to insert into the new target column.
• [x] Add logic to the SQLTranslator, SQLWatcher, and Reconciler to emit relevant span configurations.
  • https://github.com/cockroachdb/cockroach/pull/74737
  • https://github.com/cockroachdb/cockroach/pull/74803
  • https://github.com/cockroachdb/cockroach/pull/74913
  • https://github.com/cockroachdb/cockroach/pull/75122
• [x] #75688 Ensure pts reconciliation works as it did before.
• [x] #76606 Teach the SQLTranslator to emit SystemTargets
• [x] https://github.com/cockroachdb/cockroach/pull/76605
• [x] #76948 Hookup the Reconciler to use the SystemSpanConfigStore.

KV side

• [x] #74765: Data model change to span configurations to enable PTS
• [x] https://github.com/cockroachdb/cockroach/pull/76213: Introduce Targets and Records, remove roachpb.SpanConfigEntry in package spanconfig to as a prerequisite to modifying RPCs
• [x] https://github.com/cockroachdb/cockroach/pull/76219: Modify existing RPCs to get/update SystemSpanConfigs
• [x] KVAccessor changes to write system span configurations
  • https://github.com/cockroachdb/cockroach/pull/76414
  • https://github.com/cockroachdb/cockroach/pull/76721
• [x] #76871 Introduce the SystemSpanConfigStore to keep an in-memory copy of all SystemSpanConfigs + hydrate configurations when serving them
• [x] #76942 Teach the KVSubscriber to react to system span configuration updates
• [x] PTSReader interface:

  • 75285 Introduce the ProtectedTSReader interface, have the protectedts.Cache implement it, and make AdminVerifyProtectedTimestamp a no-op.

    • 77338 Make the KVSubscriber implement the ProtectedTSReader interface and create an adapter interface to work with both the old and new subsystems for v22.1.

Stability followups (requirements):

• [x] https://github.com/cockroachdb/cockroach/issues/76931
• [x] https://github.com/cockroachdb/cockroach/issues/77156
• [x] https://github.com/cockroachdb/cockroach/issues/77239
• [x] https://github.com/cockroachdb/cockroach/issues/77495
• [x] https://github.com/cockroachdb/cockroach/pull/77349 Add validation to ensure span configs associated with SystemTargets only have the ProtectionPolicy field set on them.
• [x] #75883 Remove Verify calls, and verification logic while still supporting semantics in a mixed-version state.

Stability followups (nice to have):

• [ ] Add support for tenant spans to spanconfig.Store and spanconfig.KVSubscriber datadriven tests.

Dropped approach for RPCS:

• https://github.com/cockroachdb/cockroach/pull/75615
• https://github.com/cockroachdb/cockroach/pull/75641
• https://github.com/cockroachdb/cockroach/pull/75769

Epic: CRDB-10306

Jira issue: CRDB-11712

[blathers-crl[bot]]

cc @cockroachdb/bulk-io

[blathers-crl[bot]]

Hi @adityamaru, please add branch-* labels to identify which branch(es) this release-blocker affects.

_{:owl: Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.}

[adityamaru]

PTS is now wired up end to end to work in a multi-tenant cluster. We will track follow-up release blockers in their own issues.

[arulajmani]

I think it's safe to say this thing is done, so I'm going to close out this issue.

[mari-crl]

:information_source: Hello! I am a human and not at all a robot! Look at my very human username! :robot: :notes: :thinking: Although I tried very hard to figure out what to do with this issue, more powerful human brains will need to help me. (specifically: Both Github and Jira issues were modified after exalate problems) :confounded: :arrows_counterclockwise: Please visit this issue's mirror at CRDB-11712 and try to sync the two sides up manually. :star2: :white_check_mark: When you're finished, comment saying as much asn a member of Developer Infrastructure will be along to finish linking. :link: :no_entry_sign: Note that until this is done, this issue is not and will not be synced to Jira with Exalate. :no_entry_sign: :sweat_smile: Feeling lost? Don't worry about it! A member of @cockroachdb/exalate-22-cleanup-team will be along shortly to help! :+1: :construction_worker: Developer Infrastructure members: when ready, open Exalate from the right-hand menu of the mirror issue in Jira, then choose Connect and enter this issue's URN: cockroachdb/cockroach-73727. Either way, delete this comment when you're done. :key: :pray: Thank you for your compliance, my fellow humans! :robot: :wave:

[mwang1026]

sync done

[rail]

Manually synced with Jira