cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
29.96k stars 3.79k forks source link

rangefeed: reduce goroutines for rangefeed registrations #110432

Open erikgrinaker opened 1 year ago

erikgrinaker commented 1 year ago

With the mux rangefeed protocol and the rangefeed scheduler in #107553, the only remaining O(ranges) goroutine scaling in rangefeeds is with registrations. These are basically responsible for buffering and sending events from the processor to the gRPC stream. We currently spin up one registration goroutine per range/client pair both for the plain and mux rangefeed protocols. In both cases, this is wasteful:

There is a draft PR for this in #109667, but it unfortunately won't make it in for 23.2. We should get it into 24.1 instead -- in that case, we may remove the legacy processor and only use the scheduler (and possibly even the mux), which would significantly reduce the complexity here since we won't have to support all permutations.

Touches #96395.

Jira issue: CRDB-31426

Epic CRDB-37519

blathers-crl[bot] commented 1 year ago

cc @cockroachdb/replication

blathers-crl[bot] commented 9 months ago

cc @cockroachdb/cdc

blathers-crl[bot] commented 6 months ago

Hi @exalate-issue-sync[bot], please add branch-* labels to identify which branch(es) this GA-blocker affects.

:owl: Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.