search

drmingdrmer / openraft

An implementation of the Raft distributed consensus protocol using the Tokio framework.
https://datafuselabs.github.io/openraft/
Apache License 2.0
1 stars 1 forks source link

Split metrics into data metrics and server metrics #86

Open drmingdrmer opened 9 months ago

drmingdrmer commented 9 months ago

RaftMetrics encapsulates a range of information about the internal states of a Raft node, including the server state (e.g., Leader, Follower), and data-related states such as the last log index and the last applied log index.

Applications interested in monitoring these details can subscribe to RaftMetrics to receive updates when these metrics change, with Raft::metrics().

However, some metrics, like last_log_index, are frequently updated, while others, such as current_term and state, change less often.

For applications solely concerned with server state transitions (like switching from Leader to Follower), being notified of changes to last_log_index can result in unnecessary wake-ups.

To optimize this, it would be beneficial to add two subsets of the metrics:

By segregating these metrics, applications can subscribe to the relevant subset and reduce the overhead of processing frequent updates that are not pertinent to their needs.

TODO:

Checklist - [X] Modify `openraft/src/metrics/raft_metrics.rs` ✓ https://github.com/drmingdrmer/openraft/commit/98ef2e355a70fdc24178115c9cc265598c665ff9 [Edit](https://github.com/drmingdrmer/openraft/edit/sweep/split_metrics_into_data_metrics_and_serv/openraft/src/metrics/raft_metrics.rs#L18-L69) - [ ] Running GitHub Actions for `openraft/src/metrics/raft_metrics.rs` ⋯ [Edit](https://github.com/drmingdrmer/openraft/edit/sweep/split_metrics_into_data_metrics_and_serv/openraft/src/metrics/raft_metrics.rs#L18-L69) - [ ] Modify `openraft/src/core/raft_core.rs` ▶ [Edit](https://github.com/drmingdrmer/openraft/edit/sweep/split_metrics_into_data_metrics_and_serv/openraft/src/core/raft_core.rs#L521-L551) - [ ] Running GitHub Actions for `openraft/src/core/raft_core.rs` ▶ [Edit](https://github.com/drmingdrmer/openraft/edit/sweep/split_metrics_into_data_metrics_and_serv/openraft/src/core/raft_core.rs#L521-L551) - [ ] Modify `openraft/src/core/raft_core.rs` ▶ [Edit](https://github.com/drmingdrmer/openraft/edit/sweep/split_metrics_into_data_metrics_and_serv/openraft/src/core/raft_core.rs#L1-L10) - [ ] Running GitHub Actions for `openraft/src/core/raft_core.rs` ▶ [Edit](https://github.com/drmingdrmer/openraft/edit/sweep/split_metrics_into_data_metrics_and_serv/openraft/src/core/raft_core.rs#L1-L10)
sweep-ai[bot] commented 9 months ago
Sweeping

✨ Track Sweep's progress on our progress dashboard!


50%
⚡ Sweep Basic Tier: I'm using GPT-4. You have 5 GPT-4 tickets left for the month and 3 for the day. (tracking ID: 61ec7617ff)

For more GPT-4 tickets, visit our payment portal. For a one week free trial, try Sweep Pro (unlimited GPT-4 tickets).

[!TIP] I'll email you at drdr.xp@gmail.com when I complete this pull request!

Install Sweep Configs: Pull Request

Actions (click)

GitHub Actions✓

Here are the GitHub Actions logs prior to making any changes:

Sandbox logs for ab7f309
Checking openraft/src/metrics/raft_metrics.rs for syntax errors... ✅ openraft/src/metrics/raft_metrics.rs has no syntax errors! 1/1 ✓ 
Checking openraft/src/metrics/raft_metrics.rs for syntax errors...
✅ openraft/src/metrics/raft_metrics.rs has no syntax errors!

Sandbox passed on the latest main, so sandbox checks will be enabled for this issue.

Step 1: 🔎 Searching

I found the following snippets in your repository. I will now analyze these snippets and come up with a plan.

Some code snippets I think are relevant in decreasing order of relevance (click to expand). If some file is missing from here, you can mention the path in the ticket description. https://github.com/drmingdrmer/openraft/blob/ab7f30999aa550868af870f8b14f00c610a3bada/openraft/src/metrics/raft_metrics.rs#L17-L69 https://github.com/drmingdrmer/openraft/blob/ab7f30999aa550868af870f8b14f00c610a3bada/openraft/src/core/raft_core.rs#L521-L551
I also found the following external resources that might be helpful: **Summaries of links found in the content:** https://github.com/datafuselabs/openraft/blob/147de251551c1ed364bd60bbe57047393e1d625e/openraft/src/metrics/raft_metrics.rs#L18: The page describes the `RaftMetrics` struct in the `raft_metrics.rs` file of the `openraft` repository. `RaftMetrics` encapsulates various information about the internal states of a Raft node, including the server state, last log index, last applied log index, and more. The page suggests adding two subsets of metrics: one for data-related metrics and another for server-related metrics. This would allow applications to subscribe to the relevant subset of metrics and reduce unnecessary updates. The page also proposes adding `Raft::data_metrics()` and `Raft::server_metrics()` methods to retrieve the subsets of metrics.

Step 2: ⌨️ Coding

--- 
+++ 
@@ -12,28 +12,14 @@
 use crate::StoredMembership;
 use crate::Vote;

-/// A set of metrics describing the current state of a Raft node.
+/// A set of data-related metrics describing the current state of a Raft node.
 #[derive(Clone, Debug, PartialEq, Eq)]
 #[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize), serde(bound = ""))]
-pub struct RaftMetrics
+pub struct RaftDataMetrics
 where
     NID: NodeId,
     N: Node,
 {
-    pub running_state: Result<(), Fatal>,
-
-    /// The ID of the Raft node.
-    pub id: NID,
-
-    // ---
-    // --- data ---
-    // ---
-    /// The current term of the Raft node.
-    pub current_term: u64,
-
-    /// The last accepted vote.
-    pub vote: Vote,
-
     /// The last log index has been appended to this Raft node's log.
     pub last_log_index: Option,

@@ -45,7 +31,38 @@
     pub snapshot: Option>,

     /// The last log id that has purged from storage, inclusive.
-    ///
+    pub purged: Option>,
+/// A set of server-related metrics describing the current state of a Raft node.
+#[derive(Clone, Debug, PartialEq, Eq)]
+#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize), serde(bound = ""))]
+pub struct RaftServerMetrics
+where
+    NID: NodeId,
+    N: Node,
+{
+    pub running_state: Result<(), Fatal>,
+
+    /// The ID of the Raft node.
+    pub id: NID,
+
+    /// The current term of the Raft node.
+    pub current_term: u64,
+
+    /// The last accepted vote.
+    pub vote: Vote,
+
+    /// The state of the Raft node.
+    pub state: ServerState,
+
+    /// The current cluster leader.
+    pub current_leader: Option,
+
+    /// The current membership config of the cluster.
+    pub membership_config: Arc>,
+
+    /// The replication states. It is Some() only when this node is leader.
+    pub replication: Option>,
+}
     /// `purged` is also the first log id Openraft knows, although the corresponding log entry has
     /// already been deleted.
     pub purged: Option>,

Ran GitHub Actions for 98ef2e355a70fdc24178115c9cc265598c665ff9:
• tests-feature-test (nightly, loosen-follower-log-revert): ⋯
• tests-feature-test (nightly, single-term-leader): ⋯
• openraft-test (nightly, 0, single-term-leader): ⋯
• tests-feature-test (nightly): ⋯
• openraft-feature-test (nightly, single-term-leader,serde,singlethreaded): ⋯
• openraft-feature-test (nightly, single-term-leader): ⋯
• openraft-test (nightly, 30): ⋯
• openraft-feature-test (nightly, serde): ⋯
• Build (nightly, bench,serde,bt,singlethreaded): ⋯
• openraft-test (stable, 0): ⋯
• openraft-feature-test (nightly): ⋯
• lint: ⋯
• examples (nightly, raft-kv-rocksdb): ⋯
• stores (sledstore): ⋯
• examples (nightly, raft-kv-memstore): ⋯
• external-stores (stores/rocksstore-v2): ⋯
• examples (stable, raft-kv-rocksdb): ⋯
• stores (rocksstore-compat07): ⋯
• examples (stable, raft-kv-memstore): ⋯
• stores (rocksstore): ⋯
• cluster-benchmark (cluster_benchmark): ⋯
• stores (memstore): ⋯
• openraft-test-bench (nightly): ⋯
• check-subject: ⋯

Step 3: 🔁 Code Review

Working on it...

🎉 Latest improvements to Sweep:

💡 To recreate the pull request edit the issue title or description. To tweak the pull request, leave a comment on the pull request. Join Our Discord

This is an automated message generated by Sweep AI.

github-actions[bot] commented 9 months ago

👋 Thanks for opening this issue!

Get help or engage by: