search

opencurve / curve

Curve is a sandbox project hosted by the CNCF Foundation. It's cloud-native, high-performance, and easy to operate. Curve is an open-source distributed storage system for block and shared file storage.
https://opencurve.io
Apache License 2.0
2.33k stars 522 forks source link

[curvefs] metaserver background task optimization #2555

Open wu-hanqing opened 1 year ago

wu-hanqing commented 1 year ago

Situation

CurveFS metaserver is the metadata node of the fs cluster, which stores the dentry and inode information of the file system. Use the raft algorithm to ensure consistency among three replicas and provide high service availability.

In addition to managing metadata, metaserver is also responsible for functions such as s3compact and cleaning inodes. These functions need to operate metadata, so they need to generate a request to change metadata, and then call raft for processing.

At present, these functions will be executed simultaneously on three copies of metadata. When calling the raft process, it will determine whether the current copy is a raft leader. If not, the current task will be discarded.

Due to the fact that there is only one raft leader at the same time, the operations on the other two replicas are undoubtedly redundant, wasting a large amount of CPU resources when there are many background tasks.

Expectation

The background task only starts executing after the replica is selected as the leader and is automatically canceled after the leader abdicates.

Ziy1-Tan commented 1 year ago

I want to try it. Assign me plz.

wu-hanqing commented 1 year ago

I want to try it. Assign me plz.

You can submit a rough change plan first, and then proceed to actual code development after confirmation.

ken90242 commented 1 year ago

I am a software engineer at a tech company, working on the company's product business logic. I am interested in C++ and distributed systems.

I chose this topic because I previously worked on several issues for CurveFS, and I want to learn more about Curve design and implementation details.

wsehjk commented 10 months ago

Hello, may I apply to solve this issue if this bro @ken90242 has not finished this issue over deadline or gived up. I have solved issue #2564 and associated pr #2976 is under review. I wanna contribute more to curve

Ziy1-Tan commented 10 months ago

Hello, may I apply for to solve this issue if this bro @ken90242 has not finished this issue over deadline or gived up. I have solved issue #2564 and associated pr #2976 is under review. I wanna contribute more to curve

welcome:)