OpenAtomFoundation / pikiwidb

a high-performance, large-capacity, multi-tenant, data-persistent, strong data consistency based on raft, Redis-compatible elastic KV data storage system based on RocksDB
BSD 3-Clause "New" or "Revised" License
195 stars 62 forks source link

feat: support pd #395

Closed panlei-coder closed 1 month ago

panlei-coder commented 1 month ago

1、进展:在单 raft 的基础之上支持的 pd 模块,pd 按照独立部署的形式设计(混合部署等后续功能差不多了再支持),pd 和 普通节点都默认使用的是 PRAFT 和 db 0,目前能够完成 pd 初始化为一个单 raft group 之后,普通节点启动之后会主动上报注册(需要传入 pd 的成员配置信息)。 2、后续安排: (1)把 multi-raft 的代码合并进来,完善整体的功能。 (2)目前 pd 文件夹下的 pd.proto 文件和 src 文件夹下的 store.proto 文件的 cmake 自动生成有点问题,还没来得及解决,需要大家帮忙一起看一下,在代码调试的时候我是直接手动生成的: protoc --proto_path=./src --proto_path=./src/pd --cpp_out=./cmake-build-debug/generated_pd/ pd.proto protoc --proto_path=./src --cpp_out=./cmake-build-debug/generated_pd/ store.proto (3)添加测试,目前只是简单验证了正确性。 3、测试: pd节点:默认端口为 7777,pikiwidb.conf 中 as-pd 设置为 yes 普通节点:默认端口为8888,pikiwidb.conf 中 as-pd 设置为 no 先启动 pd 节点,然后再启动普通节点。

Summary by CodeRabbit

coderabbitai[bot] commented 1 month ago

Walkthrough

此次更新对项目的构建配置和服务架构进行了显著改进。新增了对协议缓冲区(Protocol Buffers)的支持,增强了数据库管理功能,并引入了新的放置驱动服务,使得分布式数据库系统的操作更加高效和灵活。

Changes

文件 变更摘要
CMakeLists.txt, src/CMakeLists.txt, src/pd/CMakeLists.txt 添加了对新的子目录和协议缓冲区文件的支持,创建了新的静态库,并为可执行目标添加了依赖关系。
pikiwidb.conf 启用了RAFT共识算法,并调整了相关参数以增强配置功能。
src/base_cmd.cc, src/base_cmd.h 改进了命令执行逻辑,增强了错误处理和流控制。
src/cmd_raft.cc, src/cmd_raft.h 引入了数据库初始化和后端管理的新功能,增加了客户端相关的包含指令。
src/config.cc, src/config.h 添加了多个配置参数以支持新的“pd”功能,增强了PConfig类的可配置性。
src/db.cc, src/db.h 修改了数据库标识符的命名,并添加了一个新的初始化方法,增强了代码的清晰度和一致性。
src/pd/*.proto, src/pd/pd_server.cc, src/pd/pd_service.cc, src/pd/pd_service.h 新增了放置驱动服务及其相关的方法,提供对存储和区域的管理。
src/store.cc, src/store.h, src/store.proto, src/store_service.cc, src/store_service.h 引入了新的存储和区域统计管理功能,通过RPC方法提供更好的服务架构。

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant StoreService
    participant PlacementDriverService

    User->>StoreService: 请求获取存储统计
    StoreService->>PlacementDriverService: 调用GetStoreStats
    PlacementDriverService-->>StoreService: 返回存储统计
    StoreService-->>User: 返回统计信息

Poem

🐇 在代码的世界里跳跃,
新服务如花儿绽放,
统计数据轻松拿,
构建更快不再忙。
协议缓冲真方便,
兔子欢快乐无边! 🌼


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share - [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A&url=https%3A//coderabbit.ai) - [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai) - [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai) - [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai&mini=true&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)
Tips ### Chat There are 3 ways to chat with [CodeRabbit](https://coderabbit.ai): - Review comments: Directly reply to a review comment made by CodeRabbit. Example: - `I pushed a fix in commit .` - `Generate unit testing code for this file.` - `Open a follow-up GitHub issue for this discussion.` - Files and specific lines of code (under the "Files changed" tab): Tag `@coderabbitai` in a new review comment at the desired location with your query. Examples: - `@coderabbitai generate unit testing code for this file.` - `@coderabbitai modularize this function.` - PR comments: Tag `@coderabbitai` in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples: - `@coderabbitai generate interesting stats about this repository and render them as a table.` - `@coderabbitai show all the console.log statements in this repository.` - `@coderabbitai read src/utils.ts and generate unit testing code.` - `@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.` - `@coderabbitai help me debug CodeRabbit configuration file.` Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. ### CodeRabbit Commands (invoked as PR comments) - `@coderabbitai pause` to pause the reviews on a PR. - `@coderabbitai resume` to resume the paused reviews. - `@coderabbitai review` to trigger an incremental review. This is useful when automatic reviews are disabled for the repository. - `@coderabbitai full review` to do a full review from scratch and review all the files again. - `@coderabbitai summary` to regenerate the summary of the PR. - `@coderabbitai resolve` resolve all the CodeRabbit review comments. - `@coderabbitai configuration` to show the current CodeRabbit configuration for the repository. - `@coderabbitai help` to get help. Additionally, you can add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed. ### CodeRabbit Configuration File (`.coderabbit.yaml`) - You can programmatically configure CodeRabbit by adding a `.coderabbit.yaml` file to the root of your repository. - Please see the [configuration documentation](https://docs.coderabbit.ai/guides/configure-coderabbit) for more information. - If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: `# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json` ### Documentation and Community - Visit our [Documentation](https://coderabbit.ai/docs) for detailed information on how to use CodeRabbit. - Join our [Discord Community](https://discord.com/invite/GsXnASn26c) to get help, request features, and share feedback. - Follow us on [X/Twitter](https://twitter.com/coderabbitai) for updates and announcements.