search

OriginTrail / ot-node

OriginTrail Decentralized Knowledge Graph network node
https://origintrail.io
Apache License 2.0
207 stars 75 forks source link

getKnowledgeAssetsWithPagination #3336

Open botnumberseven opened 1 month ago

botnumberseven commented 1 month ago

My node is trying to sync paranet assets, I can see in logs getKnowledgeAssetsWithPagination calls which are followed by a bunch of getKnowledgeAssetLocator.

The trouble is - is restarts from the beginning when the node is restarted. So in a real life where one paranet could hold 10s or 100s of thousands of assets, this process of going thru all assets with getKnowledgeAssetsWithPagination might never finishes.

It could be several days to go thru all assets with getKnowledgeAssetsWithPagination, but if node restarts for any reason (even node update triggered by the node itself) it resets.

Could you make it that it does not start from 0 after every node restart?

Mihajlo-Pavlovic commented 1 month ago

Thanks for input, we plan to optimise this code in the future