search

Granola-Team / mina-indexer

The Mina Indexer is a re-imagined version of the software collectively called the "Mina archive node."
Apache License 2.0
16 stars 10 forks source link

Problem: "replay" gives an assertion failure #1196

Open robinbb opened 2 weeks ago

robinbb commented 2 weeks ago

Tier-3 testing repeatedly gives an assertion failure due to attempts to use 'replay' feature:

2024-06-20T23:32:49.226448-07:00 - INFO Replaying db new block (length 9989): 3NLuViLZZ36gLUDDFJ2hXqcZs2J7H1YpLk2SSfeQDWiXnwJ2wmtM
2024-06-20T23:32:49.233585-07:00 - DEBUG Best block is better than the incoming block
2024-06-20T23:32:49.233636-07:00 - INFO Replaying staking ledger (epoch 0): jx7buQVWFLsXTtzRgSxbYcT8EYLS8KCZbLrfDcJxMtyy4thw2Ee
2024-06-20T23:32:49.243206-07:00 - INFO Replaying aggregate delegations epoch 0
thread 'tokio-runtime-worker' panicked at src/state/mod.rs:1177:29:
assertion `left == right` failed
jhult commented 1 week ago

Relates to #1033

Isaac-DeFrain commented 6 days ago

Do we have evidence that this happens with consistent db versions?

robinbb commented 6 days ago

I'm not sure we have enough information to know.

robinbb commented 9 hours ago

Adding '--self-check' to the 'server start' reliably produces an assertion failure. See the commented-out code in 'ops/deploy.rb' (grep for 'self-check'). So, this problem persists, and is not related to a one-time corruption of a database directory.