input-output-hk / mithril

Stake-based threshold multi-signatures protocol
https://mithril.network
Apache License 2.0
123 stars 39 forks source link

Vacuum sqlite database at startup & after transactions preload #1771

Closed Alenar closed 3 months ago

Alenar commented 3 months ago

Content

This PR:

Pre-submit checklist

Comments

After merging #1766 we observed that the SQLite database file size did not decrease after pruning transactions in signer. This is due to SQLite not automatically reclaiming free pages by itself, we need to use the vacuum operation to do that.

Note: Vacuum reconstruct the SQLite by creating a new file, that mean that it needs an exclusive lock on the database so we can't use it after every deletion carelessly.

For everyday task we enable auto-vacuum that makes SQLite handle it by itself but a the cost of greater fragmentation. Given this drawback we only enable it on Cardano transactions databases since they are the only one were removal of data occurs on large scale, otherwise simply executing Vacuum at startup is enough.

Issue(s)

Relates to #1766

github-actions[bot] commented 3 months ago

Test Results

    4 files  ±0     51 suites  ±0   8m 48s :stopwatch: +2s 1 090 tests +1  1 090 :white_check_mark: +1  0 :zzz: ±0  0 :x: ±0  1 238 runs  +1  1 238 :white_check_mark: +1  0 :zzz: ±0  0 :x: ±0 

Results for commit 335e28dd. ± Comparison against base commit 47c2bb80.

:recycle: This comment has been updated with latest results.