search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.05k stars 3.8k forks source link

kvserver: track size of unstable raft log #99783

Open tbg opened 1 year ago

tbg commented 1 year ago

This was hacked together in #98576 but it's generally useful.

Jira issue: CRDB-26076

Epic CRDB-39958

blathers-crl[bot] commented 1 year ago

cc @cockroachdb/replication

tbg commented 1 year ago

https://github.com/cockroachdb/cockroach/issues/99464#issuecomment-1486825371 is another issue where this metric would be something I'd check if it existed.

ajd12342 commented 11 months ago

Hi @tbg ! I am Anuj Diwan, a Computer Science PhD student at UT Austin. I am part of a team along with @arjunrs1 (Arjun Somayazulu) and we're taking a graduate Distributed Systems course. For our course project, we are interested in contributing to CockroachDB. This issue is related to our course material. Could we work on this issue? Any pointers for us to get started would be appreciated as well.

Thanks and regards, Anuj.

pav-kv commented 11 months ago

Hi @ajd12342, thank you for reaching out.

Have you already contributed to CRDB? If not, consider starting from this guide.

As for this issue, you could maybe start by looking at the pkg/kv/kvserver/metrics.go file which defines a bunch of metrics. In this issue we are intrested in having something like the "raft.unstable_bytes" metric in the draft PR that is referred to in the issue description.

Then look where and how it's incremented, and try to replicate some of that (we would need it to be more polished than in that PR though).

If you're looking to understand what's behind this metric, it's the raft's unstable part of the log which sits in memory.

Let us know where this leads you, and what questions you will have.