search

cardano-foundation / cf-ledger-sync

An open-source Java application designed to facilitate access to indexed Cardano blockchain data
Apache License 2.0
17 stars 1 forks source link

Ledger sync speed #84

Open M2tec opened 7 months ago

M2tec commented 7 months ago

Testing ledger sync as backend for Gamechanger wallet. Syncing Preprod was pretty fast. But now syncing mainnet takes a long time. Sync started at 13 november at 16:00. Now at slot 56.882.115. Is this the way it should go? Or is there a snapshot like for db-sync?

System memory 128GiB System memory processor AMD Ryzen 9 7900X3D 12-Core Processor storage Corsair MP700 2TB

satran004 commented 7 months ago

Thanks @M2tec for trying out Ledger Sync.

Yes, the speed is an issue in the current version. The code in the main branch lacks batch support, leading to the slow saving of block data one at a time.

However, we have recently introduced batch support in the develop branch. Here's the PR https://github.com/cardano-foundation/cf-ledger-sync/pull/57.

Currently, we are testing this feature for mainnet sync. There are a few minor issues on mainnet that we expect to resolve soon, and we plan to merge these improvements into the main branch as quickly as possible. But syncing with all test networks are ok.

While batch storage support will enhance speed, syncing the mainnet may still take ~7 days or slightly longer.

We don't have snapshot available now. But I think we have to plan for that. I will check with our DevOps team to assess its feasibility.

Our immediate goal, post-open sourcing, is to simplify the architecture and achieve a stable build. We anticipate tagging our first stable build soon.

Then, our focus will shift to performance improvement, including exploring options like parallel processing of blocks.

M2tec commented 7 months ago

Ok thanks, that is very informative. I will keep an eye on the progress.