Armory crashes when scanning transactions with lots of addresses, transactions.

[smartbrain]

Hi,

I'd like to raise an issue. Armory crashes when restoring a certain wallet with lots of addresses and lots and lots of small transactions (about a dozen addresses and daily mining payouts for just under a month).

The problem persists on both Windows and Linux, on binaries and on Linux from the latest github version. Last night tried a fresh install with a new Bitcoin blockchain download too on Linux and that did not work.

Armory will crash when scanning the transaction history. The armorylog.txt file shows the following error:

2014-02-03 19:44 (ERROR) -- armoryengine.py:12346 - BDM was not ready for your request!  Waited 20 sec.
2014-02-03 19:44 (ERROR) -- armoryengine.py:12347 -   getattr   name: getTopBlockHeight
2014-02-03 19:44 (ERROR) -- armoryengine.py:12348 - BDM currently doing: Passthrough (70518724)
2014-02-03 19:44 (ERROR) -- armoryengine.py:12349 - Waiting for completion: ID= 54881326
2014-02-03 19:44 (ERROR) -- armoryengine.py:12350 - Direct traceback
2014-02-03 19:44 (ERROR) -- armoryengine.py:12353 - Traceback:
Traceback (most recent call last):
  File "/home/don/BitcoinArmory/armoryengine.py", line 12343, in passthruFunc
    out = self.outputQueue.get(True, self.mtWaitSec)
  File "/usr/lib/python2.7/Queue.py", line 176, in get
    raise Empty
Empty

When running from the command line (python ArmoryQt.py) the following errors appear.

(ERROR) armoryengine.py:12346 - BDM was not ready for your request!  Waited 20 sec.
(ERROR) armoryengine.py:12347 -   getattr   name: getTopBlockHeight
(ERROR) armoryengine.py:12348 - BDM currently doing: Passthrough (70518724)
(ERROR) armoryengine.py:12349 - Waiting for completion: ID= 54881326
(ERROR) armoryengine.py:12350 - Direct traceback
  File "ArmoryQt.py", line 5133, in <module>
    os._exit(QAPP.exec_())
  File "/home/don/BitcoinArmory/qt4reactor.py", line 232, in _iterate
    self.doIteration(delay, fromqt)
  File "/home/don/BitcoinArmory/qt4reactor.py", line 244, in doIteration
    self.qApp.processEvents(QEventLoop.AllEvents, delay * 1000)
  File "/home/don/BitcoinArmory/qt4reactor.py", line 103, in read
    log.callWithLogger(w, _read)
  File "/usr/lib/python2.7/dist-packages/twisted/python/log.py", line 88, in callWithLogger
    return callWithContext({"system": lp}, func, *args, **kw)
  File "/usr/lib/python2.7/dist-packages/twisted/python/log.py", line 73, in callWithContext
    return context.call({ILogContext: newCtx}, func, *args, **kw)
  File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 118, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 81, in callWithContext
    return func(*args,**kw)
  File "/home/don/BitcoinArmory/qt4reactor.py", line 102, in _read
    self.reactor._iterate(fromqt=True)
  File "/home/don/BitcoinArmory/qt4reactor.py", line 231, in _iterate
    self.runUntilCurrent()
  File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py", line 824, in runUntilCurrent
    call.func(*call.args, **call.kw)
  File "ArmoryQt.py", line 4684, in Heartbeat
    self.finishLoadBlockchain()
  File "ArmoryQt.py", line 1916, in finishLoadBlockchain
    self.walletMap[wltID].detectHighestUsedIndex(True)  # expand wlt if necessary
  File "/home/don/BitcoinArmory/armoryengine.py", line 7911, in detectHighestUsedIndex
    self.syncWithBlockchainLite(0)
  File "/home/don/BitcoinArmory/armoryengine.py", line 7213, in syncWithBlockchainLite
    self.lastSyncBlockNum = TheBDM.getTopBlockHeight(wait=True)
  File "/home/don/BitcoinArmory/armoryengine.py", line 12351, in passthruFunc
    traceback.print_stack()
(ERROR) armoryengine.py:12353 - Traceback:
Traceback (most recent call last):
  File "/home/don/BitcoinArmory/armoryengine.py", line 12343, in passthruFunc
    out = self.outputQueue.get(True, self.mtWaitSec)
  File "/usr/lib/python2.7/Queue.py", line 176, in get
    raise Empty
Empty
Segmentation fault (core dumped)
don@don-GE60-0NC-GE60-0ND:~/BitcoinArmory$ 

Python uses around 21% of memory with 1.3GB still left (all three test machines have 8 GB, have not tried 16 though).

Deleting that wallet and restoring another with only a few transactions and one address works fine.

Happy to help provide more information.

I wonder if more people will run into this problem once (if) they reach the 120+ transactions I have in this wallet.

:) Don

[gustafj]

I'm having this exact issue as well, but i don't have that many transactions and only couple of receiving addresses. I first notified the support mail about this on the 27th of January (have not received any response to that mail) and at that time i had:

• One outgoing transaction
• One receiving address
• ~48 incoming transactions

It worked on Jan 8th which was when I did that outgoing transaction, don't remember when i first noticed the problem, but probably around Jan 15th.

I believe this is the same issue as described in https://bitcointalk.org/index.php?topic=373287.0 and could also have the same root cause as #113 (It segfaults for me if I leave it running for a while, probably when a new block is seen on the network). Could be same as #170 as well.

My problematic scenario:

• Synchronizing with network: 100%
• Build databases and Scan: 100%
• "Finished blockchain scan" & "Added new blocks to memory pool" are printed to terminal
• Scanning Transaction History: 99% (15 seconds)
• Here the problem starts, UI becomes extremely slow/unresponsive and the errors starts to appear in the terminal
• "Armory is online" shown in green after a while.
• The Wallet balance changes from (...) to the current BTC amount after a few minutes.
• After a while it crashes

I'm also willing to help troubleshoot this and also try any patches if wanted.

/ Gustaf

[qubicorn]

I'm having the same issue, very similar to what Gustaf described. I'm on Windows 8. There are lots of addresses and lots of transactions.

[qubicorn]

Here is my log:

2014-02-07 21:39 (INFO) -- armoryengine.pyc:12479 - Reading blockchain, pct complete: 99.9
2014-02-07 21:39 (INFO) -- armoryengine.pyc:12479 - Reading blockchain, pct complete: 99.9
2014-02-07 21:39 (INFO) -- ArmoryQt.py:4465 - Dashboard switched to fully-online mode
2014-02-07 21:39 (INFO) -- ArmoryQt.py:3838 - Switching Armory state text to Mgmt:User, State:OnlineFull1
2014-02-07 21:39 (INFO) -- ArmoryQt.py:3780 - Switching Armory functional mode to "Online"
2014-02-07 21:39 (INFO) -- ArmoryQt.py:3838 - Switching Armory state text to Mgmt:User, State:OnlineFull2
2014-02-07 21:39 (INFO) -- ArmoryQt.py:4465 - Dashboard switched to fully-online mode
2014-02-07 21:39 (INFO) -- ArmoryQt.py:3838 - Switching Armory state text to Mgmt:User, State:OnlineFull1
2014-02-07 21:39 (INFO) -- ArmoryQt.py:3780 - Switching Armory functional mode to "Online"
2014-02-07 21:39 (INFO) -- ArmoryQt.py:3838 - Switching Armory state text to Mgmt:User, State:OnlineFull2
2014-02-07 21:39 (INFO) -- ArmoryQt.py:1913 - Syncing wallet: ##########
2014-02-07 21:39 (INFO) -- ArmoryQt.py:1913 - Syncing wallet: ##########
2014-02-07 21:39 (ERROR) -- armoryengine.pyc:12346 - BDM was not ready for your request!  Waited 20 sec.
2014-02-07 21:39 (ERROR) -- armoryengine.pyc:12347 -   getattr   name: scanRegisteredTxForWallet
2014-02-07 21:39 (ERROR) -- armoryengine.pyc:12348 - BDM currently doing: Passthrough (6824182)
2014-02-07 21:39 (ERROR) -- armoryengine.pyc:12349 - Waiting for completion: ID= 6824182
2014-02-07 21:39 (ERROR) -- armoryengine.pyc:12350 - Direct traceback
2014-02-07 21:39 (ERROR) -- armoryengine.pyc:12353 - Traceback:
Traceback (most recent call last):
  File "armoryengine.pyc", line 12343, in passthruFunc
  File "Queue.pyc", line 176, in get
Empty
2014-02-07 21:39 (ERROR) -- armoryengine.pyc:12346 - BDM was not ready for your request!  Waited 20 sec.
2014-02-07 21:39 (ERROR) -- armoryengine.pyc:12347 -   getattr   name: getTopBlockHeight
2014-02-07 21:39 (ERROR) -- armoryengine.pyc:12348 - BDM currently doing: Passthrough (6824182)
2014-02-07 21:39 (ERROR) -- armoryengine.pyc:12349 - Waiting for completion: ID= 33688535
2014-02-07 21:39 (ERROR) -- armoryengine.pyc:12350 - Direct traceback
2014-02-07 21:39 (ERROR) -- armoryengine.pyc:12353 - Traceback:
Traceback (most recent call last):
  File "armoryengine.pyc", line 12343, in passthruFunc
  File "Queue.pyc", line 176, in get
Empty
2014-02-07 21:40 (ERROR) -- armoryengine.pyc:12346 - BDM was not ready for your request!  Waited 20 sec.
2014-02-07 21:40 (ERROR) -- armoryengine.pyc:12347 -   getattr   name: scanRegisteredTxForWallet
2014-02-07 21:40 (ERROR) -- armoryengine.pyc:12348 - BDM currently doing: Passthrough (6824182)
2014-02-07 21:40 (ERROR) -- armoryengine.pyc:12349 - Waiting for completion: ID= 74787266
2014-02-07 21:40 (ERROR) -- armoryengine.pyc:12350 - Direct traceback
2014-02-07 21:40 (ERROR) -- armoryengine.pyc:12353 - Traceback:
Traceback (most recent call last):
  File "armoryengine.pyc", line 12343, in passthruFunc
  File "Queue.pyc", line 176, in get
Empty
2014-02-07 21:40 (ERROR) -- armoryengine.pyc:13289 - ErrorOut var over-represented number of errors!
2014-02-07 21:40 (ERROR) -- armoryengine.pyc:12346 - BDM was not ready for your request!  Waited 20 sec.
2014-02-07 21:40 (ERROR) -- armoryengine.pyc:12347 -   getattr   name: getTopBlockHeight
2014-02-07 21:40 (ERROR) -- armoryengine.pyc:12348 - BDM currently doing: Passthrough (74787266)
2014-02-07 21:40 (ERROR) -- armoryengine.pyc:12349 - Waiting for completion: ID= 27787670
2014-02-07 21:40 (ERROR) -- armoryengine.pyc:12350 - Direct traceback
2014-02-07 21:40 (ERROR) -- armoryengine.pyc:12353 - Traceback:
Traceback (most recent call last):
  File "armoryengine.pyc", line 12343, in passthruFunc
  File "Queue.pyc", line 176, in get
Empty
2014-02-07 21:41 (ERROR) -- armoryengine.pyc:12346 - BDM was not ready for your request!  Waited 20 sec.
2014-02-07 21:41 (ERROR) -- armoryengine.pyc:12347 -   getattr   name: getTopBlockHeight
2014-02-07 21:41 (ERROR) -- armoryengine.pyc:12348 - BDM currently doing: Passthrough (74787266)
2014-02-07 21:41 (ERROR) -- armoryengine.pyc:12349 - Waiting for completion: ID= 46528193
2014-02-07 21:41 (ERROR) -- armoryengine.pyc:12350 - Direct traceback
2014-02-07 21:41 (ERROR) -- armoryengine.pyc:12353 - Traceback:
Traceback (most recent call last):
  File "armoryengine.pyc", line 12343, in passthruFunc
  File "Queue.pyc", line 176, in get
Empty
2014-02-07 21:41 (ERROR) -- armoryengine.pyc:12346 - BDM was not ready for your request!  Waited 20 sec.
2014-02-07 21:41 (ERROR) -- armoryengine.pyc:12347 -   getattr   name: getTopBlockHeight
2014-02-07 21:41 (ERROR) -- armoryengine.pyc:12348 - BDM currently doing: Passthrough (74787266)
2014-02-07 21:41 (ERROR) -- armoryengine.pyc:12349 - Waiting for completion: ID= 63411094
2014-02-07 21:41 (ERROR) -- armoryengine.pyc:12350 - Direct traceback
2014-02-07 21:41 (ERROR) -- armoryengine.pyc:12353 - Traceback:
Traceback (most recent call last):
  File "armoryengine.pyc", line 12343, in passthruFunc
  File "Queue.pyc", line 176, in get
Empty
2014-02-07 21:41 (ERROR) -- ArmoryQt.py:4826 - Error in heartbeat function
Traceback (most recent call last):
  File "ArmoryQt.py", line 4684, in Heartbeat
  File "ArmoryQt.py", line 1921, in finishLoadBlockchain
  File "ArmoryQt.py", line 2052, in createCombinedLedger
  File "armoryengine.pyc", line 7278, in getBalance
  File "CppBlockUtils.pyc", line 1172, in getSpendableBalance
NotImplementedError: Wrong number or type of arguments for overloaded function 'BtcWallet_getSpendableBalance'.
  Possible C/C++ prototypes are:
    BtcWallet::getSpendableBalance(uint32_t)
    BtcWallet::getSpendableBalance()

2014-02-07 21:41 (ERROR) -- armoryengine.pyc:13289 - ErrorOut var over-represented number of errors!
2014-02-07 21:41 (INFO) -- ArmoryQt.py:4465 - Dashboard switched to fully-online mode
2014-02-07 21:41 (INFO) -- ArmoryQt.py:3838 - Switching Armory state text to Mgmt:User, State:OnlineFull1
2014-02-07 21:41 (INFO) -- ArmoryQt.py:3780 - Switching Armory functional mode to "Online"
2014-02-07 21:41 (INFO) -- ArmoryQt.py:3838 - Switching Armory state text to Mgmt:User, State:OnlineFull2
2014-02-07 21:41 (INFO) -- ArmoryQt.py:1913 - Syncing wallet: ##########
2014-02-07 21:41 (INFO) -- ArmoryQt.py:1913 - Syncing wallet: ##########
2014-02-07 21:42 (ERROR) -- armoryengine.pyc:12346 - BDM was not ready for your request!  Waited 20 sec.
2014-02-07 21:42 (ERROR) -- armoryengine.pyc:12347 -   getattr   name: getTopBlockHeight
2014-02-07 21:42 (ERROR) -- armoryengine.pyc:12348 - BDM currently doing: Passthrough (16535853)
2014-02-07 21:42 (ERROR) -- armoryengine.pyc:12349 - Waiting for completion: ID= 6062904
2014-02-07 21:42 (ERROR) -- armoryengine.pyc:12350 - Direct traceback
2014-02-07 21:42 (ERROR) -- armoryengine.pyc:12353 - Traceback:
Traceback (most recent call last):
  File "armoryengine.pyc", line 12343, in passthruFunc
  File "Queue.pyc", line 176, in get
Empty
2014-02-07 21:42 (ERROR) -- armoryengine.pyc:12346 - BDM was not ready for your request!  Waited 20 sec.
2014-02-07 21:42 (ERROR) -- armoryengine.pyc:12347 -   getattr   name: getTopBlockHeight
2014-02-07 21:42 (ERROR) -- armoryengine.pyc:12348 - BDM currently doing: Passthrough (16535853)
2014-02-07 21:42 (ERROR) -- armoryengine.pyc:12349 - Waiting for completion: ID= 3205558
2014-02-07 21:42 (ERROR) -- armoryengine.pyc:12350 - Direct traceback
2014-02-07 21:42 (ERROR) -- armoryengine.pyc:12353 - Traceback:
Traceback (most recent call last):
  File "armoryengine.pyc", line 12343, in passthruFunc
  File "Queue.pyc", line 176, in get
Empty
2014-02-07 21:42 (ERROR) -- armoryengine.pyc:12346 - BDM was not ready for your request!  Waited 20 sec.
2014-02-07 21:42 (ERROR) -- armoryengine.pyc:12347 -   getattr   name: getTopBlockHeight
2014-02-07 21:42 (ERROR) -- armoryengine.pyc:12348 - BDM currently doing: Passthrough (16535853)
2014-02-07 21:42 (ERROR) -- armoryengine.pyc:12349 - Waiting for completion: ID= 76600370
2014-02-07 21:42 (ERROR) -- armoryengine.pyc:12350 - Direct traceback
2014-02-07 21:42 (ERROR) -- armoryengine.pyc:12353 - Traceback:
Traceback (most recent call last):
  File "armoryengine.pyc", line 12343, in passthruFunc
  File "Queue.pyc", line 176, in get
Empty
2014-02-07 21:42 (ERROR) -- ArmoryQt.py:4826 - Error in heartbeat function
Traceback (most recent call last):
  File "ArmoryQt.py", line 4684, in Heartbeat
  File "ArmoryQt.py", line 1921, in finishLoadBlockchain
  File "ArmoryQt.py", line 2052, in createCombinedLedger
  File "armoryengine.pyc", line 7278, in getBalance
  File "CppBlockUtils.pyc", line 1172, in getSpendableBalance
NotImplementedError: Wrong number or type of arguments for overloaded function 'BtcWallet_getSpendableBalance'.
  Possible C/C++ prototypes are:
    BtcWallet::getSpendableBalance(uint32_t)
    BtcWallet::getSpendableBalance()

2014-02-07 21:43 (ERROR) -- armoryengine.pyc:13289 - ErrorOut var over-represented number of errors!
2014-02-07 21:43 (ERROR) -- armoryengine.pyc:12346 - BDM was not ready for your request!  Waited 20 sec.
2014-02-07 21:43 (ERROR) -- armoryengine.pyc:12347 -   getattr   name: hasTxWithHash
2014-02-07 21:43 (ERROR) -- armoryengine.pyc:12348 - BDM currently doing: Passthrough (65813534)
2014-02-07 21:43 (ERROR) -- armoryengine.pyc:12349 - Waiting for completion: ID= 13792294
2014-02-07 21:43 (ERROR) -- armoryengine.pyc:12350 - Direct traceback
2014-02-07 21:43 (ERROR) -- armoryengine.pyc:12353 - Traceback:
Traceback (most recent call last):
  File "armoryengine.pyc", line 12343, in passthruFunc
  File "Queue.pyc", line 176, in get
Empty
2014-02-07 21:43 (ERROR) -- armoryengine.pyc:12346 - BDM was not ready for your request!  Waited 20 sec.
2014-02-07 21:43 (ERROR) -- armoryengine.pyc:12347 -   getattr   name: hasTxWithHash
2014-02-07 21:43 (ERROR) -- armoryengine.pyc:12348 - BDM currently doing: Passthrough (65813534)
2014-02-07 21:43 (ERROR) -- armoryengine.pyc:12349 - Waiting for completion: ID= 54557274
2014-02-07 21:43 (ERROR) -- armoryengine.pyc:12350 - Direct traceback
2014-02-07 21:43 (ERROR) -- armoryengine.pyc:12353 - Traceback:
Traceback (most recent call last):
  File "armoryengine.pyc", line 12343, in passthruFunc
  File "Queue.pyc", line 176, in get
Empty
2014-02-07 21:44 (ERROR) -- armoryengine.pyc:12346 - BDM was not ready for your request!  Waited 20 sec.
2014-02-07 21:44 (ERROR) -- armoryengine.pyc:12347 -   getattr   name: hasTxWithHash
2014-02-07 21:44 (ERROR) -- armoryengine.pyc:12348 - BDM currently doing: Passthrough (65813534)
2014-02-07 21:44 (ERROR) -- armoryengine.pyc:12349 - Waiting for completion: ID= 60777217
2014-02-07 21:44 (ERROR) -- armoryengine.pyc:12350 - Direct traceback
2014-02-07 21:44 (ERROR) -- armoryengine.pyc:12353 - Traceback:
Traceback (most recent call last):
  File "armoryengine.pyc", line 12343, in passthruFunc
  File "Queue.pyc", line 176, in get
Empty
2014-02-07 21:44 (ERROR) -- armoryengine.pyc:13289 - ErrorOut var over-represented number of errors!
2014-02-07 21:44 (INFO) -- ArmoryQt.py:4465 - Dashboard switched to fully-online mode
2014-02-07 21:44 (INFO) -- ArmoryQt.py:3838 - Switching Armory state text to Mgmt:User, State:OnlineFull1
2014-02-07 21:44 (INFO) -- ArmoryQt.py:3780 - Switching Armory functional mode to "Online"
2014-02-07 21:44 (INFO) -- ArmoryQt.py:3838 - Switching Armory state text to Mgmt:User, State:OnlineFull2
2014-02-07 21:44 (INFO) -- ArmoryQt.py:1913 - Syncing wallet: ##########
2014-02-07 21:44 (INFO) -- ArmoryQt.py:1913 - Syncing wallet: ##########
2014-02-07 21:44 (ERROR) -- ArmoryQt.py:4826 - Error in heartbeat function
Traceback (most recent call last):
  File "ArmoryQt.py", line 4684, in Heartbeat
  File "ArmoryQt.py", line 1921, in finishLoadBlockchain
  File "ArmoryQt.py", line 2052, in createCombinedLedger
  File "armoryengine.pyc", line 7278, in getBalance
  File "CppBlockUtils.pyc", line 1172, in getSpendableBalance
NotImplementedError: Wrong number or type of arguments for overloaded function 'BtcWallet_getSpendableBalance'.
  Possible C/C++ prototypes are:
    BtcWallet::getSpendableBalance(uint32_t)
    BtcWallet::getSpendableBalance()

2014-02-07 21:44 (ERROR) -- armoryengine.pyc:12346 - BDM was not ready for your request!  Waited 20 sec.
2014-02-07 21:44 (ERROR) -- armoryengine.pyc:12347 -   getattr   name: hasTxWithHash
2014-02-07 21:44 (ERROR) -- armoryengine.pyc:12348 - BDM currently doing: Passthrough (129221)
2014-02-07 21:44 (ERROR) -- armoryengine.pyc:12349 - Waiting for completion: ID= 36309857
2014-02-07 21:44 (ERROR) -- armoryengine.pyc:12350 - Direct traceback
2014-02-07 21:44 (ERROR) -- armoryengine.pyc:12353 - Traceback:
Traceback (most recent call last):
  File "armoryengine.pyc", line 12343, in passthruFunc
  File "Queue.pyc", line 176, in get
Empty

[rxgrant]

186 should fix this. Please confirm.

[qubicorn]

My issue turned out to be a corrupt wallet.

[gustafj]

No longer any issue for me after 0.91 beta! ;)