More recent snapshots (2.4.1)

[pablosymc]

Hi, thanks so much for this awesome tool! This is kind of a selfish request, but I was wondering if we could get some more recent snapshots for (2.4.1)? Look's like the snapshots from 2.3.4 start at slot ~81m and it would just honestly just make syncing much quicker since we're at slot ~97m now?

Or if you have advice on how I can generate my own snapshots, I could also attempt that as well. Thanks so much for your time!

[KtorZ]

Snapshots are just made from running Kupo with a wildcard pattern. I have been a bit lazy to redo more recent snapshots for latest releases. Ideally, I would setup a nightly build for that to keep snapshots in sync.

[pablosymc]

Thanks @KtorZ Do you mind sharing the commands you used if possible?

[KtorZ]

kupo \
  --node-config $CFG/$NETWORK/cardano-node/config.json \
  --node-socket /tmp/node.socket \
  --workdir $HOME/.cache/kupo/$NETWORK \
  --since origin \
  --match '*' \
  --defer-db-indexes \
  --prune-utxo

That's usually what I run for snapshots ☝️. With NETWORK and CFG being set to whatever they need to.

[pablosymc]

kupo \
  --node-config $CFG/$NETWORK/cardano-node/config.json \
  --node-socket /tmp/node.socket \
  --workdir $HOME/.cache/kupo/$NETWORK \
  --since origin \
  --match '*' \
  --defer-db-indexes \
  --prune-utxo

That's usually what I run for snapshots ☝️. With NETWORK and CFG being set to whatever they need to.

I was referring to the commands used to generate the snapshots :) Let's say I'm at the tip, and now want to create snapshots from where I'm at. What's the best way to go about that, so incase I need to do a quick restore I can?

Is it just a simple tar gz zip of the kupo.sqlite3 file (chunked into multiple files)?

[KtorZ]

Is it just a simple tar gz zip of the kupo.sqlite3 file (chunked into multiple files)?

Indeed! Here are the steps I usually go through (assuming a kupo.sqlite3 in .)

1. Vacuum the database

sqlite3 ./kupo.sqlite3
> VACUUM;
> PRAGMA OPTIMIZE;

2. Make an archive (tarball) + compress it with gzip:

GZIP=-9 tar cvzf kupo.sqlite3-$NETWORK.tar.gz kupo.sqlite3

3. Calculate a checksum of the archive with md5

md5 kupo.sqlite3-$NETWORK.tar.gz

4. If > 500 MB, split the archive in chunks <= 500 MB

split -b 500M kupo.sqlite3-$NETWORK.tar.gz

And that's it. Could be nice to add those steps to the Makefile now that I think about it 🤔 (e.g. make snapshot)

[pablosymc]

Exactly what I was looking for @KtorZ !! Thank you so much for your help and your time!