eigerco / lumina

Wasm friendly Celestia light node implementation in Rust
Apache License 2.0
116 stars 33 forks source link

Optimize batch inserts and retrievals in Store. #282

Open zvolin opened 4 months ago

zvolin commented 4 months ago

Currently Store trait requires methods like

    async fn append_single_unchecked(&self, header) -> Result<()>;
    async fn get_header_by_height(&self, height) -> Result<ExtendedHeader>;

and based on those it provides sth like

    async fn append_sunchecked(&self, headers: Vec<_>) -> Result<()>;
    async fn get_headers_range(&self, range: impl Range) -> Result<ExtendedHeader>;

We should reverse this, having the bulk methods required so that they can be optimized, and provide the methods for single units, just wrapping them in vec's or similar.

Also, in IndexedDb, on appending we do uniqueness checks even tho we have .unique() flags on the indexes. This sounds like a double checks, so one of those could be dropped

fl0rek commented 1 month ago

TODO: