eqlabs / pathfinder

A Starknet full node written in Rust
https://eqlabs.github.io/pathfinder/
Other
619 stars 227 forks source link

feat(websocket): broadcast events from the pending block #2062

Closed sistemd closed 2 months ago

sistemd commented 3 months ago

Closes https://github.com/eqlabs/pathfinder/issues/2019.

kkovaacs commented 3 months ago

Hmm, how does this handle the case where the pending block itself is updated? (Ie. the same block is updated on the watch channel multiple times, each time new transactions/receipts/events are added.)

Mirko-von-Leipzig commented 3 months ago

This is going to be quite tricky to get robustly correct. The pending data watcher has very few guarantees of sequentialliaty - e.g. if the pending polling fails for a few times or takes long, its entirely possible to skip an entire block. Which I think would break your implementation.

I think it should be possible to make something robust if we mark our last processed event and ensure we only process sequentially forwards.

// Track the block number and event index of the last event processed.
// Maybe `next_xxx` is a better name
let mut current_block = storage.latest + 1;
let mut event_count = 0;

select! {
    pending => {
        if pending.block_number == current_block && pending.events.len() > event_count {
            emit_events(&pending.events[event_count..]);
            event_count = pending.events.len() + 1;
        }
    },
    full_block => {
            // Don't forget to check full_block.events.len() > event_count here.
            emit_events(&full_block.events[event_count..]);
            current_block = full_block.number + 1;
            event_count = 0;
    }
}

This achieves a few things

  1. We only allow pending events from the pending block if its the current block being processed.
  2. We always wait for the block to be closed before moving onto the next one.
  3. We only process new events
sistemd commented 2 months ago

@Mirko-von-Leipzig https://github.com/eqlabs/pathfinder/pull/2062/commits/f8f0688f7d601088f7e228044308e5285fa08d7e