kroma-network / go-ethereum

go-ethereum for Kroma
GNU Lesser General Public License v3.0
48 stars 23 forks source link

feat: state migration and transition from ZKT to MPT #121

Closed Pangssu closed 5 days ago

Pangssu commented 3 weeks ago

Migrate the backend of state trie from ZK Trie to MPT for better performance and user experience, and to enable zkVM fault proof. This PR also involves reducing code and state differences with op-geth.

This PR should be merged after #118 is merged.

coderabbitai[bot] commented 3 weeks ago

[!IMPORTANT]

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (2) * dev * main

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

The changes in this pull request enhance the Ethereum Geth client by adding new configuration options and flags related to the Kroma protocol, specifically for managing MPT migration and ZK features. Key modifications include updates to configuration handling, the introduction of a state migrator for transitioning from ZKTrie to MPT, and enhancements to error handling and logging. Additionally, new test cases validate the functionality of self-destruct operations and the behavior of the state migrator. The overall structure and logic of the application are preserved while integrating these new features.

Changes

File Change Summary
cmd/geth/config.go Added DisableMPTMigration field and new command-line flags OverrideKromaMPT, KromaZKTrie. Updated error handling in loadConfig and modified dumpConfig. Removed duplicate imports.
cmd/geth/main.go Added utils.OverrideKromaMPT flag and reintroduced utils.RollupSequencerHTTPFlag.
cmd/utils/flags.go Added flags OverrideKromaMPT, DisableMPTMigrationFlag, and reintroduced RollupSequencerHTTPFlag. Removed unsupported flags.
consensus/beacon/consensus.go Modified IsTTDReached to always return true for Kroma. Updated comments for clarity.
core/blockchain.go Introduced GetMigratedRef method. Updated InsertChain and writeBlockWithState for MPT handling.
core/blockchain_reader.go Enhanced HasState method to handle errors when opening a trie for Kroma.
core/blockchain_test.go Added TestSelfDestructReEnabled and modified TestSelfDestructDisabled for Kroma MPT checks.
core/genesis.go Added OverrideKromaMPT field to ChainOverrides struct. Updated SetupGenesisBlockWithOverride.
core/kroma_migration.go Introduced MigratedRef struct and methods for managing migration references.
core/types/transaction.go Added IsSystemTx method to Transaction struct.
core/vm/instructions.go Added uint256 import. Updated opSelfdestruct and opSelfdestruct6780 for Kroma checks.
core/vm/interpreter.go Removed specific handling for SELFDESTRUCT opcode in Kroma. Enhanced tracing capabilities.
eth/api_backend.go Added HistoricalRPCService method to EthAPIBackend struct.
eth/backend.go Introduced migrator field in Ethereum struct and added StateMigrator method.
eth/catalyst/api.go Updated forkchoiceUpdated and NewPayload methods for Kroma state migration checks.
eth/ethconfig/config.go Added OverrideKromaMPT and DisableMPTMigration fields. Removed rollup-related fields.
eth/ethconfig/gen_config.go Enhanced MarshalTOML and UnmarshalTOML methods with new fields for Kroma.
eth/tracers/api.go Added HistoricalRPCService() method. Updated error handling in tracing methods.
eth/tracers/api_test.go Introduced mockHistoricalBackend for testing historical RPC functionality.
eth/tracers/native/prestate.go Updated CaptureStart to include new Kroma addresses for account lookups.
internal/ethapi/api.go Removed unsupported historical RPC handling code.
internal/ethapi/api_test.go Updated error handling in TestCall function for non-existent blocks.
internal/ethapi/backend.go Added HistoricalRPCService() method to Backend interface.
internal/ethapi/transaction_args_test.go Added HistoricalRPCService method to backendMock struct.
migration/migrator.go Introduced StateMigrator struct for managing state migrations.
migration/migrator_genesis.go Added zkPreimageWithAlloc function for retrieving preimages from the genesis block.
migration/migrator_newstate.go Implemented methods for managing state transitions and updates.
migration/migrator_range.go Introduced hashRange struct for managing hash ranges during migrations.
migration/migrator_validate.go Added ValidateMigratedState method for validating migrated state consistency.
miner/miner.go Added BackendWithHistoricalState interface for historical state retrieval.
miner/payload_building.go Modified buildPayload to handle Kroma migration blocks.
miner/worker.go Updated makeEnv and generateWork for Kroma state retrieval logic.
params/config.go Added KromaMPTTime and Optimism fields to ChainConfig.
rpc/errors.go Introduced NoHistoricalFallbackError type for error handling.
trie/iterator.go Updated seek method to accept zk.TreePath. Added new helper methods for traversal.
trie/iterator_test.go Enhanced test coverage for trie iterator functionality.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant GethClient
    participant StateMigrator
    participant Database

    User->>GethClient: Start migration with new flags
    GethClient->>StateMigrator: Initialize migration
    StateMigrator->>Database: Retrieve current state
    Database-->>StateMigrator: Return current state
    StateMigrator->>StateMigrator: Process state transitions
    StateMigrator->>Database: Update migrated state
    Database-->>StateMigrator: Confirm update
    StateMigrator-->>GethClient: Migration complete
    GethClient-->>User: Migration successful

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share - [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A&url=https%3A//coderabbit.ai) - [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai) - [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai) - [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai&mini=true&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)
🪧 Tips ### Chat There are 3 ways to chat with [CodeRabbit](https://coderabbit.ai): - Review comments: Directly reply to a review comment made by CodeRabbit. Example: - `I pushed a fix in commit , please review it.` - `Generate unit testing code for this file.` - `Open a follow-up GitHub issue for this discussion.` - Files and specific lines of code (under the "Files changed" tab): Tag `@coderabbitai` in a new review comment at the desired location with your query. Examples: - `@coderabbitai generate unit testing code for this file.` - `@coderabbitai modularize this function.` - PR comments: Tag `@coderabbitai` in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples: - `@coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.` - `@coderabbitai read src/utils.ts and generate unit testing code.` - `@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.` - `@coderabbitai help me debug CodeRabbit configuration file.` Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. ### CodeRabbit Commands (Invoked using PR comments) - `@coderabbitai pause` to pause the reviews on a PR. - `@coderabbitai resume` to resume the paused reviews. - `@coderabbitai review` to trigger an incremental review. This is useful when automatic reviews are disabled for the repository. - `@coderabbitai full review` to do a full review from scratch and review all the files again. - `@coderabbitai summary` to regenerate the summary of the PR. - `@coderabbitai resolve` resolve all the CodeRabbit review comments. - `@coderabbitai configuration` to show the current CodeRabbit configuration for the repository. - `@coderabbitai help` to get help. ### Other keywords and placeholders - Add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed. - Add `@coderabbitai summary` to generate the high-level summary at a specific location in the PR description. - Add `@coderabbitai` anywhere in the PR title to generate the title automatically. ### CodeRabbit Configuration File (`.coderabbit.yaml`) - You can programmatically configure CodeRabbit by adding a `.coderabbit.yaml` file to the root of your repository. - Please see the [configuration documentation](https://docs.coderabbit.ai/guides/configure-coderabbit) for more information. - If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: `# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json` ### Documentation and Community - Visit our [Documentation](https://docs.coderabbit.ai) for detailed information on how to use CodeRabbit. - Join our [Discord Community](http://discord.gg/coderabbit) to get help, request features, and share feedback. - Follow us on [X/Twitter](https://twitter.com/coderabbitai) for updates and announcements.
Pangssu commented 3 weeks ago

@coderabbitai review

coderabbitai[bot] commented 3 weeks ago
:white_check_mark: Actions performed Review triggered. > Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.
seolaoh commented 1 week ago

After Kroma MPT time, if a user wants to sync their node from genesis without snapshot, that user might have zkt db and mpt db from genesis to Kroma MPT time. What do you think about this case? Should we highly recommend to sync with snapshot? Or we need to handle this case like starting state migrator only when the block height reaches some days before Kroma MPT time?

Pangssu commented 1 week ago

After Kroma MPT time, if a user wants to sync their node from genesis without snapshot, that user might have zkt db and mpt db from genesis to Kroma MPT time. What do you think about this case? Should we highly recommend to sync with snapshot? Or we need to handle this case like starting state migrator only when the block height reaches some days before Kroma MPT time?

Users can sync their nodes from genesis and the migration will be seamless. Using snapshots will help them sync faster, but we don't have to force it.