Automatic Reduction of -dbcache Setting after Initial Sync in Bitcoin Core

[BenWestgate]

This issue proposes implementing an automatic reduction of the -dbcache setting in Bitcoin Core after the initial sync is completed. Currently, using a larger than default dbcache can lead to extended shutdown times, which can be inconvenient for users. The suggestion is to decrease the -dbcache setting to the default by not specifying it as a launch parameter once the sync is caught up.

The system should not automatically restart bitcoin-qt, but use a lower value on the next start by the user. The proposed value for the reduction of -dbcache setting is to the default, offering a faster shutdown and optimized performance. This enhancement will also address issue #41.

This means the initial -dbcache setting can be the maximum that will not cause an OOM error or significant lag in #41 while a laggy experience or long shutdown time will resolve after sync completes.

https://www.reddit.com/r/Bitcoin/comments/l95p42/bitcoin_core_still_taking_a_long_time_to_sync/

having a larger than default dbcache makes it take a very long time to shut down, as a result you should probably only increase it during your initial sync then reduce it after.

You can leave it at 7GB while you're doing the initial sync, then shut down (which will take 20 minutes or so) and remove the setting so it just uses the default.

Don't restart, just start it up and let it run overnight. Restarting it might momentarily cause a bit more apparent progress but will slow it down overall.

Based on these comments by Gregory Maxwell. The -dbache setting should be reduced by bitcoin-qt-wrapper when the sync is caught up. This should be no further than about 2 weeks or a month of blocks blocks considering my other recommendation to log into Bails at least weekly to not forget its passphrase. A long very long shutdown time every week would make this annoying and be a bad user experience.

The system should NOT restart bitcoin-qt automatically to reduce the -dbcache unless the user restarts it.

Related to #41

[BenWestgate]

https://github.com/BenWestgate/Bails/issues/41#issuecomment-1656035960

Lagginess has been resolved.

[github-actions[bot]]

Stale issue message

[BenWestgate]

@epiccurious settings.json is not supposed to be written to while open. But when bitcoin starts it doesn't know the progress because we don't persist debug.log.

Should we use a shutdown command to remove dbcache from settings.json when the node is synced and shutdown?

Should Bitcoin Core shutdown automatically and restart with default dbcache if the node is synched and user has been inactive for an hour? (For example after IBD completes in the middle of the night, saves the user a long shutdown when they return)

Or vice versa, if user has been inactive for 5 minutes, shutdown and restart with full dbcache on if it finds itself far behind?

It can't immediately restart as that would mess up uri handling and be a bad UX.

I already added code to turn full dbcache on if the node hadn't synched for 2 weeks. But that's not a full solution to this issue. We need a way to turn it off.

[BenWestgate]

The suggestion is to decrease the -dbcache setting to the default by not specifying it as a launch parameter once the sync is caught up.

We are no longer using launch parameters to manage these settings but are editing the settings.json file. This file should not be modified while bitcoin core is running. So lets create an exit script for bitcoin core to turn this off when the program is stopped.

I also think the dbcache should be reduced to default settings automatically after the IBD finishes so the user doesn't have to wait around after they hit 100% to create their backup or shutdown the node.

[BenWestgate]

I inspected the debug.log again and even though we don't persist it, it records the opening and closing of wallet files. This presents a privacy leak that could be used to defeat the panic mode. #81

We need to investigate what log features record the system that last ran Bitcoin and the wallet activity and deactivate them in the Bails bitcoin.conf debug settings.

This might allow persisting it safely. Which gives a way to detect the progress before Bitcoin starts. However those entries also report the cache use and flushes which can reveal the memory size of the system last used to sync when the dbcache is increased.

That may also need to be turned off if possible or only logged while the default dbcache is being used.

There is a 25MB ram consumption of keeping the log in tmpfs so performance would be slightly better if we can put it on the disk. Even if that has to be inside an encrypted loopback file to not persist it.