search

eqlabs / pathfinder

A Starknet full node written in Rust
https://eqlabs.github.io/pathfinder/
Other
622 stars 231 forks source link

demo: p2p source abstraction #2076

Closed Mirko-von-Leipzig closed 2 months ago

Mirko-von-Leipzig commented 3 months ago

This is intended as a possible blueprint for abstracting over p2p sources. The current issue is that there are many corner cases in how raw p2p streams are transformed into a stream of block's of items.

Ideally we would have a single source of truth which does this robustly. This PR is an attempt at implementing this for transactions -- however it should be possible to extrapolate this to all streams (maybe excluding headers - but since those are already "blockified" they matter less).

This PR introduces three things, the first two being fairly minor but still nice to consider

  1. PartialReceipt as a wrapper around Receipt. p2p receipts aren't complete ito information as we require the transaction index - which is only available after blockification. However, constantly adding new types is cumbersome, so this is a neat way of "hiding" the data until it is completed.
  2. limit the stream from p2p client. I think ideally this would only perform a single stream attempt from one random peer. This moves the retry logic etc out to sync which simplifies things.
  3. A single TransactionSource which can be spawned as either tracking or checkpoint mode, utilising the same underlying code.

Elaborating on (3) as that's the actual core idea

It should be possible to abstract out all transaction specific parts into one or more traits. Similarly, (1) should also be abstractable in the same way - the only real difference is which protocol is used - just create a StreamKind enum and abstract over that.

I would suggest excluding headers, though it should also be possible to shoehorn them in here.. but probably not needed.

Something to also consider is that something like the state diffs probably want a different output format than Vec<T> -- that could be built into the abstraction, or just handled by the following stage. Similarly, the counting will need to be specialised, I don't think state diff counts are a 1-1 mapping. But this is easily solved via fn count(&T) -> usize method.

Example of the abstraction trait

trait Source {
   type Input;

   // Defines the stream string to pass to p2p.
   const KIND: StreamKind;

   fn count(&Self::Input) -> usize;
}