osmosis-labs / osmosis

The AMM Laboratory
https://app.osmosis.zone
Apache License 2.0
893 stars 595 forks source link

feat: [indexer] optimization - publish token supplies and pool pair data after synced #8744

Closed cryptomatictrader closed 1 month ago

cryptomatictrader commented 1 month ago

This commit:

  1. Promotes the node status checker from sqs level to common level, so that it is used by both sqs and indexer ingesters.
  2. Modifies the fullIndexerBlockProcessStrategy in indexer such that it delays the publishing of token supplies and pool pair data until the node sync finishes (i.e., when catch up is false).
  3. Adds test cases for the full_indexer_block_process_strategy::ProcessBlock function:
    1. When the node has caught up and is no longer syncing:
      • Publishes the correct number of token supplies and offsets.
      • Publishes the correct number of pool pairs, along with the expected number of pools with creation data.
      • Returns an error if it fails to verify the node's syncing status.
    2. When the node is still syncing:
      • Returns an error, and no data should be published.
github-actions[bot] commented 1 month ago

Important Notice

This PR modifies an in-repo Go module. It is one of:

The dependent Go modules, especially the root one, will have to be updated to reflect the changes. Failing to do so might cause e2e to fail.

Please follow the instructions below:

  1. Open https://github.com/osmosis-labs/osmosis/actions/workflows/go-mod-auto-bump.yml
  2. Provide the current branch name
  3. On success, confirm if an automated commit corretly updated the go.mod and go.sum files

Please let us know if you need any help.

coderabbitai[bot] commented 1 month ago

Walkthrough

The pull request introduces significant changes to the OsmosisApp implementation by integrating a new service for node status checking from the commonservice module. The nodeStatusChecker variable is updated to use this new service, and method signatures are adjusted accordingly. Additionally, there are modifications to package names, transitioning from commondomain to domain in several files, while maintaining existing logic and functionality. New interfaces and structs are introduced, enhancing the overall structure without altering the core behavior of the application.

Changes

File Path Change Summary
app/app.go Updated nodeStatusChecker to use commonservice.NewNodeStatusChecker(rpcAddress); modified indexerStreamingService initialization to include nodeStatusChecker.
ingest/common/domain/errors.go Changed package name from domain to commondomain.
ingest/common/domain/block_pools.go Changed package name from commondomain to domain.
ingest/common/domain/block_process_strategy_manager.go Changed package name from commondomain to domain.
ingest/common/domain/block_process_strategy_manager_test.go Changed package name from commondomain_test to domain_test.
ingest/common/domain/block_update_process_utils.go Changed package name from commondomain to domain; added BlockUpdateProcessUtilsI interface and BlockUpdateProcessUtils struct with methods.
ingest/common/domain/keepers.go Changed package name from commondomain to domain.
ingest/common/domain/pools.go Changed package name from commondomain to domain; corrected method signature typo in PoolExtractor interface.
ingest/common/domain/process_strategy.go Changed package name from commondomain to domain.
ingest/indexer/service/blockprocessor/full_indexer_block_process_strategy.go Added nodeStatusChecker field; modified ProcessBlock to include synchronization checks.
ingest/indexer/service/indexer_streaming_service.go Added nodeStatusChecker field; modified constructor to include this parameter.

Possibly related PRs

Suggested reviewers


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share - [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A&url=https%3A//coderabbit.ai) - [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai) - [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai) - [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai&mini=true&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)
🪧 Tips ### Chat There are 3 ways to chat with [CodeRabbit](https://coderabbit.ai): - Review comments: Directly reply to a review comment made by CodeRabbit. Example: - `I pushed a fix in commit , please review it.` - `Generate unit testing code for this file.` - `Open a follow-up GitHub issue for this discussion.` - Files and specific lines of code (under the "Files changed" tab): Tag `@coderabbitai` in a new review comment at the desired location with your query. Examples: - `@coderabbitai generate unit testing code for this file.` - `@coderabbitai modularize this function.` - PR comments: Tag `@coderabbitai` in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples: - `@coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.` - `@coderabbitai read src/utils.ts and generate unit testing code.` - `@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.` - `@coderabbitai help me debug CodeRabbit configuration file.` Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. ### CodeRabbit Commands (Invoked using PR comments) - `@coderabbitai pause` to pause the reviews on a PR. - `@coderabbitai resume` to resume the paused reviews. - `@coderabbitai review` to trigger an incremental review. This is useful when automatic reviews are disabled for the repository. - `@coderabbitai full review` to do a full review from scratch and review all the files again. - `@coderabbitai summary` to regenerate the summary of the PR. - `@coderabbitai resolve` resolve all the CodeRabbit review comments. - `@coderabbitai configuration` to show the current CodeRabbit configuration for the repository. - `@coderabbitai help` to get help. ### Other keywords and placeholders - Add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed. - Add `@coderabbitai summary` to generate the high-level summary at a specific location in the PR description. - Add `@coderabbitai` anywhere in the PR title to generate the title automatically. ### CodeRabbit Configuration File (`.coderabbit.yaml`) - You can programmatically configure CodeRabbit by adding a `.coderabbit.yaml` file to the root of your repository. - Please see the [configuration documentation](https://docs.coderabbit.ai/guides/configure-coderabbit) for more information. - If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: `# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json` ### Documentation and Community - Visit our [Documentation](https://coderabbit.ai/docs) for detailed information on how to use CodeRabbit. - Join our [Discord Community](http://discord.gg/coderabbit) to get help, request features, and share feedback. - Follow us on [X/Twitter](https://twitter.com/coderabbitai) for updates and announcements.
cryptomatictrader commented 1 month ago

Closing this PR to avoid some complicated merge issues and also to make running local test easier. Opened this PR to extend this one.