cosmos / iavl

Merkleized IAVL+ Tree implementation in Go
Apache License 2.0
409 stars 256 forks source link

chore: cleanup unnecessary `ndb.Commit` calls #902

Closed cool-develope closed 3 months ago

cool-develope commented 4 months ago

Context

There are some unnecessary ndb.Commit and resetBatch calls which introduced to avoid the large batch commit originally. They are not necessary since we are using BatchWithFlusher now.

coderabbitai[bot] commented 4 months ago

Walkthrough

The recent modifications streamline the codebase by simplifying commit processes and batch operation logic. The changes remove unnecessary variables and parameters such as commitGap and isGenesis, and eliminate batch-related functions, thus enhancing code clarity and reducing complexity. These adjustments aim to improve the efficiency of node saving and version handling, particularly in the context of fast node additions and database operations, making the system more straightforward and less error-prone.

Changes

Files Change Summary
mutable_tree.go Removed commitGap and associated logic, streamlined SaveVersion by removing isGenesis, simplified saveFastNodeAdditions.
mutable_tree_test.go Adjusted test expectations for database operations, batch writes, and closures, updated test cases for storage upgrade.
nodedb.go Removed resetBatch and writeBatch functions, simplified SaveNode method by eliminating batch operation conditions.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share - [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A&url=https%3A//coderabbit.ai) - [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai) - [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai) - [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai&mini=true&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)

Tips ### Chat There are 3 ways to chat with CodeRabbit: > Note: Auto-reply has been disabled for this repository by the repository owner. The CodeRabbit bot will not respond to your replies unless it is explicitly tagged. - Files and specific lines of code (under the "Files changed" tab): Tag `@coderabbitai` in a new review comment at the desired location with your query. Examples: - `@coderabbitai generate unit tests for this file.` - `@coderabbitai modularize this function.` - PR comments: Tag `@coderabbitai` in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples: - `@coderabbitai generate interesting stats about this repository and render them as a table.` - `@coderabbitai show all the console.log statements in this repository.` - `@coderabbitai read src/utils.ts and generate unit tests.` - `@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.` Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. ### CodeRabbit Commands (invoked as PR comments) - `@coderabbitai pause` to pause the reviews on a PR. - `@coderabbitai resume` to resume the paused reviews. - `@coderabbitai review` to trigger a review. This is useful when automatic reviews are disabled for the repository. - `@coderabbitai resolve` resolve all the CodeRabbit review comments. - `@coderabbitai help` to get help. Additionally, you can add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed. ### CodeRabbit Configration File (`.coderabbit.yaml`) - You can programmatically configure CodeRabbit by adding a `.coderabbit.yaml` file to the root of your repository. - The JSON schema for the configuration file is available [here](https://coderabbit.ai/integrations/coderabbit-overrides.v2.json). - If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: `# yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json` ### CodeRabbit Discord Community Join our [Discord Community](https://discord.com/invite/GsXnASn26c) to get help, request features, and share feedback.
tac0turtle commented 4 months ago

backport?

cool-develope commented 3 months ago

backport?

not sure, @czarcas7ic osmosis implemented the db wrapper with flusher fully? I think we can merge it, and later backport when need

czarcas7ic commented 3 months ago

@cool-develope I didn't understand how to implement in v0.37, see https://osmosis-network.slack.com/archives/C04LXMASL0P/p1708037594466689?thread_ts=1707244984.568649&channel=C04LXMASL0P&message_ts=1708037594.466689

czarcas7ic commented 3 months ago

Thanks for implementing this for our sdk fork @cool-develope! Seems we are unblocked to merge so we can test

czarcas7ic commented 3 months ago

Sorry maybe I am not fully understanding how implementing with flusher has to do with this PR?

In the PR you made @cool-develope, it seems like NewBatch and NewBatchWithSize still behave the same, which I thought that was what needed to be addressed https://github.com/osmosis-labs/cosmos-sdk/blob/6c1422877673277448bc14e2be415133cc6738e6/store/wrapper/tmdb.go#L52-L57

My apologies if my understanding is off here.

cool-develope commented 3 months ago

Sorry maybe I am not fully understanding how implementing with flusher has to do with this PR?

In the PR you made @cool-develope, it seems like NewBatch and NewBatchWithSize still behave the same, which I thought that was what needed to be addressed https://github.com/osmosis-labs/cosmos-sdk/blob/6c1422877673277448bc14e2be415133cc6738e6/store/wrapper/tmdb.go#L52-L57

My apologies if my understanding is off here.

NewBatchSize is aimed to allocate the memory when create the batch, it will allow to avoid memory grow while batch insertion. It is not directly related to the flusher. The purpose of flusher is to avoid the large batch writing, it writes periodically if the batch size is greater than the threshold. Ofc, NewBatchSize will increase the performance, unfortunately I have no idea of how to refactor the prefixdb of comet-db. However, we can merge it without NewBatchSize still. You don't need to refactor anything for this PR since nodeDB internally use BatchWithFlusher, just merge it (https://github.com/osmosis-labs/cosmos-sdk/pull/582) and update the iavldb with this commit, then check if the batch writing is triggered periodically.

czarcas7ic commented 3 months ago

Thanks for the explanation!

cool-develope commented 3 months ago

@Mergifyio backport release/v1.1.x

mergify[bot] commented 3 months ago

backport release/v1.1.x

✅ Backports have been created

* [#921 chore: cleanup unnecessary `ndb.Commit` calls (backport #902)](https://github.com/cosmos/iavl/pull/921) has been created for branch `release/v1.1.x`