search

dipdup-io / dipdup

Modular framework for creating selective indexers and featureful backends for dapps
https://dipdup.io
MIT License
93 stars 52 forks source link

Optimize TzKT big map requests #656

Open droserasprout opened 1 year ago

droserasprout commented 1 year ago

Optimize tezos tzkt data source, use bigmap ptr filtration where possible. https://api.tzkt.io/#tag/BigMaps

Instead of

contract.in=KT1GBZmSxmnKJXGMdMLbugPfLyUPmuLSMwKS&path.in=store.expiry_map,store.records

Do

bigmap.in=1264,1262
Josh-121 commented 1 month ago

I am applying to this issue via OnlyDust platform.

My background and how it can be leveraged

software engineering/ smart contract dev background and contributor in similar issues from last hackthons.(https://app.onlydust.com/u/Josh-121)

How I plan on tackling this issue

Review Current Implementation:

Analyze how the current API queries are structured using contract.in and path.in to retrieve data from the Tezos blockchain. Specifically, look at calls related to the contract KT1GBZmSxmnKJXGMdMLbugPfLyUPmuLSMwKS and its paths (store.expiry_map and store.records). Switch to BigMap Pointer Filtration:

Replace the contract.in and path.in method with bigmap.in=1264,1262, directly using the BigMap pointers for more efficient queries. This reduces unnecessary overhead by bypassing contract and path lookups, directly accessing the relevant BigMap data.

Update API Requests:

Refactor API calls to use the new filtration method, ensuring the query is streamlined and optimized for performance by fetching only the required data. Testing:

Test the API requests with BigMap pointers to ensure they return the correct data and are significantly faster than the previous implementation. Validate the data accuracy compared to the original method.

Work on feedback/recommendations

Josh-121 commented 3 weeks ago

Hello @droserasprout , would push PR soon. Sorry for initial delays