search

iotaledger / inx-chronicle

IOTA permanode implemented using the IOTA Node Extension (INX) interface.
Apache License 2.0
14 stars 12 forks source link

Random Chronicle crash #1380

Open shufps opened 2 months ago

shufps commented 2 months ago

We have the problem that Chronicle randomly crashes with this error:

inx-chronicle    | 2024-04-04T16:08:51.074357Z ERROR handle_ledger_update{slot_index=11590 created=0 consumed=0}: inx_chronicle::inx: error=missing consumed output (0x348d2dd7fba634d8b5ea52b3ae16bf1c3ac6bdd227ba786c9a004f59565aeb1e332d00000000) in slot 11590
inx-chronicle    | 2024-04-04T16:08:51.074535Z ERROR inx_chronicle: a worker failed with error: missing consumed output (0x348d2dd7fba634d8b5ea52b3ae16bf1c3ac6bdd227ba786c9a004f59565aeb1e332d00000000) in slot 11590
inx-chronicle    | Error: missing consumed output (0x348d2dd7fba634d8b5ea52b3ae16bf1c3ac6bdd227ba786c9a004f59565aeb1e332d00000000) in slot 11590
inx-chronicle    | Location:
inx-chronicle    |     /inx-chronicle/src/analytics/mod.rs:352:20

It might be some issue with the ordering how data is stored into the database and that there is no guarantee that INX-streams are in sync to each other.

When deleting the database, resyncing works. But it can take a couple of hours afterwards that it crashes again.

The database is then broken und recovery is not possible