optimize gossamer archive node size - Githubissues

ChainSafe / gossamer

🕸️ Go Implementation of the Polkadot Host

https://chainsafe.github.io/gossamer

GNU Lesser General Public License v3.0

427 stars 110 forks source link

optimize gossamer archive node size #1454

Open noot opened 3 years ago

noot commented 3 years ago

Task summary

currently the gossamer kusama archive node disk size is larger than expected, at 1.3mil blocks the size of the DB is 34GB, but the size of a substrate kusama archive node at 1.6mil blocks is 15-20GB
optimize this to be the same size or less than substrate
potential solution is adding compression to the database layer, currently there is no compression

opts.Compression = options.Snappy

Also, we should update chaindb to allow for badger options to be passed from gossamer (via a config)

dutterbutter commented 3 years ago

https://wiki.polkadot.network/docs/en/maintain-sync#:~:text=However%2C%20an%20archive%20node%20takes,was%20around%2015%20to%2020GB. @noot