OpenFarcaster / teleport

A fast implementation of a Farcaster Hub, in Rust.
MIT License
57 stars 5 forks source link

eth: start_block recovery after crashing during sync #22

Open gregfromstl opened 7 months ago

gregfromstl commented 7 months ago

Right now, we're fetching the last indexed block by finding the max block number in chain_events. Because we insert log types sequentially, there's a chance the program crashes after inserting some log types for a stretch of blocks but not all, leaving the other log types absent when the sync restarts and fetches the latest block number.

The easiest way to solve this is probably by getting the max block number in chain_events for each event type. However, this will make the log collection messier (we'd be fetching logs for different ranges for each event type, since we don't want any duplicate events), so I'm not 100% convinced this is the best approach.

haardikk21 commented 7 months ago

what if we:

  1. change all INSERT queries to be upsert (INSERT ... ON CONFLICT ...) instead
  2. and, fetch start_block as the minimum value of latest block of all event types
gregfromstl commented 7 months ago

I think that would work, we would just have to make the transaction_hash column unique in chain_events to prevent duplicates there

haardikk21 commented 6 months ago

i can take this