search

IntersectMBO / cardano-db-sync

A component that follows the Cardano chain and stores blocks and transactions in PostgreSQL
Apache License 2.0
290 stars 162 forks source link

Allow electing number of ledger-state snapshots retained on disk #1813

Open rdlrt opened 1 month ago

rdlrt commented 1 month ago

OS Your OS: N/A

Versions The db-sync version (eg cardano-db-sync --version): 13.4.0.1 PostgreSQL version: 16.1.x.x

Build/Install Method The method you use to build or install cardano-db-sync: cabal

Run method The method you used to run cardano-db-sync (eg Nix/Docker/systemd/none): systemd

Additional context

Each snapshot seems to take ~2.6GB now on mainnet - and we seem to have 10 versions saved. I assume once saved - these are only required when restarting dbsync and using the last known snapshot to resume ledger-state from(?)

ls -lrt ../guild-db/ledger-state/
# total
# -rw-r--r--  2589239888  Jul  18  21:49  129772796-ad1a1974ed-497.lstate
# -rw-r--r--  2593990097  Jul  23  21:48  130204797-fc2f5a85ec-498.lstate
# -rw-r--r--  2591911078  Jul  28  21:48  130636777-c1ecf5af89-499.lstate
# -rw-r--r--  2594919050  Aug  2   21:48  131068792-128d08577d-500.lstate
# -rw-r--r--  2599942347  Aug  7   21:48  131500793-3a3135e237-501.lstate
# -rw-r--r--  2599814216  Aug  12  16:48  131914806-37afa6610a.lstate
# -rw-r--r--  2599702507  Aug  12  19:45  131925476-4f08b814d7.lstate
# -rw-r--r--  2599768309  Aug  12  21:48  131932767-0e5d2fb39b-502.lstate
# -rw-r--r--  2513408568  Aug  12  22:28  131935227-8796fb5477.lstate
# -rw-r--r--  2513694475  Aug  13  01:16  131945325-a08e58266b.lstate

Problem Report

Provide ability to limit number of ledger state snapshots that are being retained by dbsync - maybe as a config, and [optionally] use much lower defaults.