search

taikoxyz / taiko-mono

A based rollup. 🥁
https://taiko.xyz
MIT License
4.43k stars 2.08k forks source link

feat(relayer): two-step bridge + watchdog + full merkle proof #15669

Closed cyberhorsey closed 2 months ago

cyberhorsey commented 3 months ago

1) Update the relayer indexer subcommand to be able to index either MessageSent or MessageReceived, add the event name to the queue name to have separate queues for each flow. We will now deploy two relayer indexers for each source-dest chain, to feed either queue (to be either Processed, or checked if they need to be Suspended).

2) Update the relayer processor to be able to handle the the new two-step bridge, as well as full merkle proofs: if a proofReceipt exists onchain, we are in the second step, if it doesnt, we are in the first step. The processor needs to be able to wait after sending the "MessageReceive" call if we are in the first step, and wait to finalize it. Ifi t is in the "MessageExecuted" step, the relayer can return early and acknowledge the queue message.

If a relayer exits early after processing a call in the first step, the message needs to be unacknowledged and go back onto the queue, to repeat the process, except detect it now has a ProofReceipt and can be processed without a proof.

3) Add a new watchdog subcommand, which reads from the new MessageReceived queue, and calls isMessageSent on the source bridge. If it exists, acknowledge the queue message succesfully. If it doesn't exist, we need to transact suspend message on chain for this transaction, and add that to the database for querying via a UI.

4) Add a new migration, repository, interface, implementation, and route to the relayer API subcommand for: GET /suspendedTransactions - list all suspended transactions in a paginated response.

5) increment a Prometheus counter when we suspend, so we can use this for an alert to alert David and I.

6) Update proof changes with latest signal service/bridge code.

TODO:

Summary by CodeRabbit

vercel[bot] commented 3 months ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
bridge-ui-v2-a6 ✅ Ready (Inspect) Visit Preview Feb 7, 2024 8:38pm
codecov[bot] commented 3 months ago

Codecov Report

Attention: Patch coverage is 19.69197% with 730 lines in your changes are missing coverage. Please review.

Project coverage is 25.73%. Comparing base (f53766b) to head (bf7d0f9). Report is 110 commits behind head on main.

:exclamation: Current head bf7d0f9 differs from pull request most recent head 0c55e79. Consider uploading reports for the commit 0c55e79 to get more accurate results

Files Patch % Lines
packages/relayer/watchdog/watchdog.go 5.73% 229 Missing and 1 partial :warning:
packages/relayer/processor/process_message.go 30.99% 103 Missing and 15 partials :warning:
...s/relayer/indexer/handle_message_received_event.go 0.00% 78 Missing :warning:
packages/relayer/indexer/indexer.go 1.42% 69 Missing :warning:
packages/relayer/indexer/save_event_to_db.go 0.00% 59 Missing :warning:
packages/relayer/indexer/subscribe.go 16.94% 45 Missing and 4 partials :warning:
packages/relayer/pkg/proof/encoded_signal_proof.go 30.61% 34 Missing :warning:
packages/relayer/watchdog/config.go 45.16% 32 Missing and 2 partials :warning:
packages/relayer/pkg/utils/gas.go 25.71% 25 Missing and 1 partial :warning:
...kages/relayer/pkg/encoding/encode_storage_proof.go 0.00% 15 Missing :warning:
... and 5 more
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #15669 +/- ## ========================================== - Coverage 26.82% 25.73% -1.10% ========================================== Files 105 113 +8 Lines 6188 6870 +682 ========================================== + Hits 1660 1768 +108 - Misses 4348 4912 +564 - Partials 180 190 +10 ``` | [Flag](https://app.codecov.io/gh/taikoxyz/taiko-mono/pull/15669/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=taikoxyz) | Coverage Δ | | *Carryforward flag | |---|---|---|---| | [eventindexer](https://app.codecov.io/gh/taikoxyz/taiko-mono/pull/15669/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=taikoxyz) | `17.22% <ø> (ø)` | | Carriedforward from [5b3595d](https://app.codecov.io/gh/taikoxyz/taiko-mono/commit/5b3595d22e9c3e042dcdc05c7bf7731b545dde6a?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=taikoxyz) | | [guardian-prover-health-check](https://app.codecov.io/gh/taikoxyz/taiko-mono/pull/15669/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=taikoxyz) | `34.81% <ø> (ø)` | | Carriedforward from [5b3595d](https://app.codecov.io/gh/taikoxyz/taiko-mono/commit/5b3595d22e9c3e042dcdc05c7bf7731b545dde6a?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=taikoxyz) | | [relayer](https://app.codecov.io/gh/taikoxyz/taiko-mono/pull/15669/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=taikoxyz) | `32.77% <19.69%> (-5.11%)` | :arrow_down: | | *This pull request uses carry forward flags. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=taikoxyz) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

cyberhorsey commented 3 months ago

@dantaik @Brechtpd is calling "bridge.isMessageSent" on the source chain a good enough "watchdog check" when we get a MessageReceived event on the destination chain?

Or should I actually look up in the database that we found an existing MessageSent event from the source chain, as well?

Brechtpd commented 3 months ago

@dantaik @Brechtpd is calling "bridge.isMessageSent" on the source chain a good enough "watchdog check" when we get a MessageReceived event on the destination chain?

Or should I actually look up in the database that we found an existing MessageSent event from the source chain, as well?

I think they are pretty much identical really looking at the code because done directly together, and the isMessageSent code is very easy so extremely low chance there could be some bug related with just that code. So depending on isMessageSent sounds good to me.

cyberhorsey commented 3 months ago

@dantaik @Brechtpd is calling "bridge.isMessageSent" on the source chain a good enough "watchdog check" when we get a MessageReceived event on the destination chain? Or should I actually look up in the database that we found an existing MessageSent event from the source chain, as well?

I think they are pretty much identical really looking at the code because done directly together, and the isMessageSent code is very easy so extremely low chance there could be some bug related with just that code. So depending on isMessageSent sounds good to me.

OK sounds good, thats what I'm doing already, so this should be sufficient. When we deploy an internal devnet we can see how it operates.

coderabbitai[bot] commented 3 months ago

Walkthrough

The updates focus on refining the ABI definitions across various vaults (ERC1155, ERC20, ERC721) and enhancing cross-chain synchronization and signal service functionalities. New features include message handling, proof retrieval, and suspended transactions management. The relayer's efficiency and functionality are improved through restructured event indexing, simplified ABI definitions, and the introduction of watchdog capabilities for monitoring and processing messages between Ethereum chains.

Changes

Files Change Summary
.../erc1155vault/ERC1155Vault.go
.../erc20vault/ERC20Vault.go
.../erc721vault/ERC721Vault.go		 Updated ABI definitions and adjusted event/function signatures.
.../icrosschainsync/ICrossChainSync.go
.../signalservice/SignalService.go		 Renamed fields, streamlined ABI definitions, and updated events.
.../bridge.go
.../processor/...
.../watchdog/watchdog.go		 Enhanced message handling, proof retrieval, and watchdog functionality.
.../cmd/flags/...
.../indexer/config.go
.../event.go		 Introduced new flags, EventName configurations, and variables.
.../migrations/...
.../pkg/repo/suspended_transaction.go
.../suspended_transaction.go		 Managed suspended transactions with new SQL script and repository.
.../pkg/encoding/...
.../pkg/proof/...		 Improved encoding and proof handling with updated structs and tests.
.../pkg/http/...
.../pkg/mock/...
.../pkg/utils/gas.go		 Added HTTP routes, mock methods, and gas-related operations.
.../prometheus.go Introduced a new metric for tracking suspended transactions.

"In the digital realm, where code does weave,
A rabbit hopped, changes to conceive.
🐰 With every line, a new world takes shape,
Bonds of chains, in ether, escape.
Watchdogs guard, while vaults align,
Across the blocks, our paths entwine."

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share - [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A&url=https%3A//coderabbit.ai) - [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai) - [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai) - [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai&mini=true&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)
Tips ### Chat There are 3 ways to chat with CodeRabbit: - Review comments: Directly reply to a review comment made by CodeRabbit. Example: - `I pushed a fix in commit .` - `Generate unit-tests for this file.` - Files and specific lines of code (under the "Files changed" tab): Tag `@coderabbitai` in a new review comment at the desired location with your query. Examples: - `@coderabbitai generate unit tests for this file.` - `@coderabbitai modularize this function.` - PR comments: Tag `@coderabbitai` in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples: - `@coderabbitai generate interesting stats about this repository from git and render them as a table.` - `@coderabbitai show all the console.log statements in this repository.` - `@coderabbitai read src/utils.ts and generate unit tests.` - `@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.` Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. ### CodeRabbit Commands (invoked as PR comments) - `@coderabbitai pause` to pause the reviews on a PR. - `@coderabbitai resume` to resume the paused reviews. - `@coderabbitai review` to trigger a review. This is useful when automatic reviews are disabled for the repository. - `@coderabbitai resolve` resolve all the CodeRabbit review comments. - `@coderabbitai help` to get help. Additionally, you can add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed. ### CodeRabbit Configration File (`.coderabbit.yaml`) - You can programmatically configure CodeRabbit by adding a `.coderabbit.yaml` file to the root of your repository. - The JSON schema for the configuration file is available [here](https://coderabbit.ai/integrations/coderabbit-overrides.v2.json). - If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: `# yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json` ### CodeRabbit Discord Community Join our [Discord Community](https://discord.com/invite/GsXnASn26c) to get help, request features, and share feedback.
cyberhorsey commented 3 months ago

Lots of files for bridge-ui-v2 that should not be committed. I guess because dave renamed the package to bridge-ui.

Gotcha, bad merge from main then. I will remove the whole folder!

dantaik commented 3 months ago

@cyberhorsey shall we merge this PR?

cyberhorsey commented 3 months ago

@cyberhorsey shall we merge this PR?

No, let me deploy it and work out any kinks today.

cyberhorsey commented 3 months ago

FYI This is just waiting for some changes I've asked Ruby to do with regards to ERC20/721/1155s (now that we have a generic invocation method, the previous method of checking which type of token we have no longer works).