search

etotheipi / BitcoinArmory

Python-Based Bitcoin Software
Other
825 stars 621 forks source link

Memory leak: Scanning block chain takes 1.7GB RAM #57

Open ghost opened 11 years ago

ghost commented 11 years ago

Testing under windows,, going online causes the Armour process to balloon to over 1.7GB RAM. I assume there is an optimization that could be made here so not everything has to be processed in RAM.

Using version 0.87.2

etotheipi commented 11 years ago

Not a bug. It's a feature! A legacy of the days when the blockchain was 300 MB...

"Persistent blockchain management" is my top priority after this major upgrade I'm working on right now. The load time will go down, RAM usage will go down, a variety of bugs will disappear. It'll use extra HDD space, but it's a good trade.

orclev commented 10 years ago

So, Armory is currently eating about 60% of my RAM all by itself sitting around 4.5G. This is frankly, unacceptable. The website was last updated 7 months ago saying that excessive RAM usage was a known issue, but a fix was expected sometime in the next few weeks. Here we are 7 months later, and still no progress. Last commit I see on here was from about a month ago. Is this still being worked on? Can we expect a fix sometime soon? I really don't want to try to find a new bitcoin client, but with RAM usage like this I may not have a choice.

etotheipi commented 10 years ago

http://www.coindesk.com/bitcoin-wallet-armory-raises-600k-seed-funding/ https://bitcointalk.org/index.php?topic=299684.0

All updates have been on the "testing" branch and it's almost done. Starting a company has been remarkably distracting along with training new people. And this was no trivial upgrade. Please be patient, it's coming. (and download the testing version and help test it for me -- there's some known usability bugs, but those should be resolved soon)

orclev commented 10 years ago

Thanks for the speedy update, I'll see about getting the testing version installed. Do I need to uninstall the current version first or can I "upgrade" over the top of it?

Update: I downloaded the windows version from the link you provided. Unfortunately it's the 32 bit version as that's all I could find, but so far it looks promising. It's still in the process of building the databases, but it hasn't cracked 1G of RAM yet which is HUGE improvement over what I was seeing in 0.88.1. So long as it runs stable (had a lot of crashing issues with pre-0.88 versions) I'm fairly happy with what I'm seeing so far.

coldmug commented 10 years ago

Awesome! I might actually download and run the testing version. I have been waiting a long time for it! Good job!

LJNielsenDk commented 10 years ago

It'll use extra HDD space, but it's a good trade.

Except for those of us with very little HDD space, fx. laptops that only have an SSD. I'm assuming this is the doubling of HDD requirement that is about to come with the 0.90 version. I have a 128 GB SSD in my laptop, I don't have this kind of space to use on duplication; the blockchain itself is pushing it.

As is in 0.88 it takes a weekend or so for my laptop to get through scanning, but there must be some middle ground, like making a note of which block was last scanned for which keys.

goatpig commented 10 years ago

"but there must be some middle ground, like making a note of which block was last scanned for which keys."

That's the plan in the future. The current RAM reduced implementation is for full nodes. We do intent to move on to light nodes at some point.

Right now you can use the --dbdir command line argument to give Armory a custom database path (like a usb harddrive)

etotheipi commented 10 years ago

Actually, the "--dbdir" argument is not hooked up. There's not currently a way to separate your databases dir from your homedir (--datadir). Want to fix that for 0.91.

And yes, the plan is to later make a lighter database, even in "full" mode. It's just that doing it this way was far easier for the first shot at getting the DB working and stable, and will actually be used in some environments anyway (i.e. when we make a super-node version of Armory).