kvserver: Evaluate the cost of checking whether a span config update is a PTS update before enqueuing

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.

https://www.cockroachlabs.com

Other

29.84k stars 3.77k forks source link

kvserver: Evaluate the cost of checking whether a span config update is a PTS update before enqueuing #111837

Closed shralex closed 9 months ago

shralex commented 11 months ago

We've recently made a change where we only enqueue updates that aren't a PTS change. The remaining question is the cost of this check.

Jira issue: CRDB-32087

blathers-crl[bot] commented 11 months ago

Hi @shralex, please add branch-* labels to identify which branch(es) this release-blocker affects.

_{:owl: Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}

andrewbaptist commented 9 months ago

The initial test on this was not looking good - there was a big performance hit when making a "no-op" span config change on a system with 2M ranges. I didn't capture CPU and it was hard to reproduce.

andrewbaptist commented 9 months ago

I reran this test on a cluster without rangefeeds and running at a "normal" usage level. I grabbed a few CPU profiles before and after running the change, and watched the latency. There was no noticeable change in P99 latency and the CPU profiles looked good.