Laggy when unlocking the screen during IBD

[BenWestgate]

One tester thought his PC had frozen and considered restarting it which would have likely corrupted his IBD progress.

The formula used for setting dbcache is picking values that cause 1300+MiB of swapping on 4 and 8 GB Memory.

The core developers do not recommend increasing beyond the default -dbcache=450 on a 4GB system, but my formula is setting 550-850 when Bitcoin-Qt is started.

As a first step, lets decrease dbcache by 100-400MiB so the system response is a bit quicker.

[BenWestgate]

This may reduce memory usage to add to my bitcoin-qt-wrapper

#!/usr/bin/env bash
export MALLOC_ARENA_MAX=1
bitcoind

"Systems with 1GB (or less) should consider reducing this default"

[BenWestgate]

Pushed initial fix to use 100MiB less dbcache: https://github.com/BenWestgate/Bails/commit/7e022b54e7b93ac9ec5564cfd0811d657e17cf84

[BenWestgate]

@epiccurious can confirm if the issue is resolved on his USB stick. I do not have a problem (8-9 seconds max latency) in my virtual machines.

The target response time is under 10 seconds without the system making visual progress towards doing what you asked of it.

If we can count to 10, before things move along, we have a problem that may lead users to restart their Bails and corrupt the download.

It is fine if unlocking the screen takes longer than 10 seconds, as long as it's not stuck on any given part without making progress for more than 10 seconds. It is NOTfine if it takes 5 minutes to unlock the screen, because then the screen would immediately relock if they weren't swirling the cursor while they waited.

This is an IBD progress issue mostly, users usually only check once a day that their node is still syncing. So it's okay for that check to take a minute instead of 2 seconds. Especially once they're expecting it to be laggy.

[BenWestgate]

I would highly recommend that you do not set a dbcache at all as 4 GB is really not enough RAM to handle a larger dbcache, Bitcoin Core's other RAM usage, operating system RAM usage, and whatever else may be running on your machine.

https://bitcoin.stackexchange.com/questions/89100/bitcoind-node-crashes-after-using-dbcache-option

Note: Free memory will always reach <5% on a system with 8GB or less RAM because the chainstate folder is cached into memory. So we set it as a constant subtracted from available ram when Core is started and accept that the whole chainstate may not always be able to fit into memory and hit the USB harder with random reads for chainstate data rather than use a smaller -dbcache and make more random writes due to flushing.

This isn't guaranteed to be faster because chainstate can be random read faster from RAM than disk but it is guaranteed to be on the safe side regarding USB wear due to excessive writes.

[BenWestgate]

Pushed initial fix to use 100MiB less dbcache: 7e022b5

Tester results:

The change you made to lower dbcache by 100mb is working well for me, almost too well. I’m wondering if it makes sense to bump it back up by 50mb

My systems are closer to 2-3 seconds to turn on now, which is possibly too responsive

Making a note for issue #58 to back off by 50MiB for the max value to strike a better balance between responsiveness and sync speed and USB wear.