stampchain-io / btc_stamps

Bitcoin Stamps Indexer
https://stampchain.io
GNU Affero General Public License v3.0
12 stars 4 forks source link

Utilize new CP v2 API endpoints OR multi-block get on v1 #260

Open reinamora137 opened 4 months ago

reinamora137 commented 4 months ago

will be interesting to see if this changes performance and perhaps we can move to multi-block downloads in one api call. currently we are pulling each block individually because prior tests created mismatched data in the mult-block query for some reason. it was never determined where that was coming from.

Note: spoke with Adam at CP, and the prior versions of CP were not handling the pagination correctly in the API. this is why we were not getting the correct stamps in the multi block calls and coded in single block calling in a multithreaded way.. This can be optimized by utilizing the get_blocks in large groups which will speed up parsing time dramatically as well.

reinamora137 commented 3 months ago

it appears that get_blocks does not allow a list of blocks to fetch all at once so perhaps utilizing the v1 API and updating to pull multiple blocks since they have fixed the pagination issues is the most efficient method. It's unclear how the caching on the server is handled for v1 without more investigation however.

https://ouzielslama.docs.apiary.io/#/reference/blocks/get-issuances-by-block/get-issuances-by-block/200?mc=reference%2Fblocks%2Fget-issuances-by-block%2Fget-issuances-by-block%2F200