search

bitcoindevkit / rust-electrum-client

Bitcoin Electrum client library. Supports plaintext, TLS and Onion servers.
MIT License
78 stars 63 forks source link

Test to showcase high memory consumption of `batch_script_get_history` #89

Open da-kami opened 1 year ago

da-kami commented 1 year ago

When running batch_script_get_history on a Linux machine with processing many scripts the memory consumption is enormous and does not go down. This test showcases this behavior.

When triggering the test with 7000 scripts we see a memory consumption of about 4GB. Note that you can add a loop to re-trigger the batching to see that the memory does not get freed.

This problem could be reproduced on multiple Linux machine, but not on MacOS.

notmandatory commented 10 months ago

Thanks for test to repro this issue, any ideas why it might be happening? And do you think this would be a common usecase? In most cases I'd expect this to be used with smaller single users wallets and for very larger server wallets something like the bitcond RPC client (or a future BIP 157/158 client).

LLFourn commented 10 months ago

Memory never being freed is an issue no matter what size though. I feel like there is a lot to improve in the way we are using electrum at every level. I know @evanlinjin has ideas.