Balances command Killed

[Aness-s]

I tried running both rusty-blockparser unspentcsv and rusty-blockparser balances and it runs for about 10 minutes then at the 31 000 block mark it terminates without any error message. I know my chain is intact as I ran a bitcoin sync this morning and have the up to date chain. I was also able to run the csvdump command with no issue, only having problems with the balances and unspentcsv ones. I was sure to use the command correctly and provided a destination path as well.

Here is the command line before it terminates INFO - parser: Status: 318137 Blocks processed. (remaining: 490460, speed: 61 blocks/s) INFO - parser: Status: 318753 Blocks processed. (remaining: 489844, speed: 61 blocks/s) Killed

[gcarq]

Can you check the system logs and memory pressure when it gets killed? Seems like you ran out of memory and the process got killed by the OOM Killer. That would explain why csvdump had no issue (because it doesn't need to hold anything in memory), but balances and unspentcsv got killed, because they have to keep all UTXOs in memory until it finds the matching UTXO that spents the old one.

[Aness-s]

You're right, I checked the system logs and I saw an out of memory oom kill message. Any way I could solve/get around this?

[gcarq]

On linux you can work around this by creating a large enough swap (32GiB). I'm not sure if there is an equivalent on windows though.

[gcarq]

Yes I think 32GiB are necessary, from the README:

The required memory usage depends on the used callback:

    simplestats: ~100MB
    csvdump: ~100M
    unspentcsvdump: ~18GB
    balances: ~18GB

NOTE: Those values are taken from parsing to block height 639631 (17.07.2020).

[gcarq]

Yes this is due to the swap file unfortunately

[Aness-s]

Thanks for your help