near / near-indexer-for-explorer

Watch NEAR network and store all the data from NEAR blockchain to PostgreSQL database
https://near-indexers.io/docs/projects/near-indexer-for-explorer
GNU General Public License v3.0
123 stars 56 forks source link

Optimize parent receipts tracking #229

Closed frol closed 2 years ago

frol commented 2 years ago

Until this point in time we maintained stateless indexing, where no information was passed from one block handler to another, but with the recent performance degradation I believe it is time to introduce some caching layer to potentially avoid SELECTing from the database in the most cases. Currently, the major contributor to the IO wait is the query that tries to find the parent receipt/transaction:

https://github.com/near/near-indexer-for-explorer/blob/1a0739806f6ef605dd6cf77744c2bdfbc07bd941/src/db_adapters/receipts.rs#L199-L224

If we maintain the cache of the parent receipt/transaction, and only fallback to SELECTing form the database when we don't have the information in our cache, we will gain quite some performance boost.