StackTrace when running yield generator

[RHavar]

2016-11-19 05:00:04,417 [MCThread    ] [INFO ]  <<pubmsg on CgAn: nick=J5iweygvSSV378dO message=!orderbook
Exception in thread MCThread:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/root/joinmarket/joinmarket/message_channel.py", line 31, in run
    self.mc.run()
  File "/root/joinmarket/joinmarket/irc.py", line 448, in run
    self.__handle_line(line)
  File "/root/joinmarket/joinmarket/irc.py", line 326, in __handle_line
    self.__handle_privmsg(_chunks[0], _chunks[2], get_irc_text(line))
  File "/root/joinmarket/joinmarket/irc.py", line 278, in __handle_privmsg
    self.on_pubmsg(nick, message)
  File "/root/joinmarket/joinmarket/message_channel.py", line 889, in on_pubmsg
    self.on_orderbook_requested(nick, self)
  File "/root/joinmarket/joinmarket/message_channel.py", line 556, in on_orderbook_requested_trigger
    self.on_orderbook_requested(nick, mc)
  File "/root/joinmarket/joinmarket/maker.py", line 277, in on_orderbook_requested
    self.msgchan.announce_orders(self.orderlist, nick, mc)
  File "/root/joinmarket/joinmarket/message_channel.py", line 285, in announce_orders
    cmd = orderlist[0]['ordertype']
IndexError: list index out of range

[AdamISZ]

Thanks for the report. Which yield generator script are you running? Also please confirm whether you are running the latest version (0.2.2 = current master branch).

Your bot's orderlist (we should really call it "offerlist" now, but same thing) was empty, this shouldn't ever happen. I haven't seen it before. The stack trace is useful, thanks, but unfortunately it can't show directly why the orderlist is empty.

First thought (apart from checking which script is involved) is it might be to do with a too-small amount of coins available. On startup this results in a "Not enough funds" message and quit. If, during running, you participated in a join which was a large enough percentage of your total available, and all the other mixdepths are empty perhaps, then while that transaction was in an unconfirmed state, those coins are not available, and the amount left might be too small for the bot to allow an offer/order to be created; this might cause such a condition, I'd have to look into it.

Anyone else have another theory?

[chris-belcher]

Could you use the scrub-log.py script found in logs/ (and also https://github.com/JoinMarket-Org/joinmarket/blob/master/logs/scrub-log.py) to remove private information from your log file and then post it here?

[AdamISZ]

In yg-pe.py (the others I think are going to be the same): in on_tx_unconfirmed, create_my_orders is called; if there are no (or insufficient) funds, then [] is returned as the to_announce list. This means that your bot is, as long as the join is unconfirmed, in a state where it has no active orders, i.e. orderlist is [].

In a sense, this is just unnecessary details; basically, if your orderlist is empty, then on_orderbook_requested should not bother attempting to announce your orders/offers, there are none. The only nuance is, while this is a crash condition on startup, i.e. you should not start your yg with no funds, it is emphatically not a crash condition during normal operation, it just means you're waiting for confirmation.

I think that's right.

[RHavar]

I was running "yield-generator-basic.py" after I had just sent 1 bitcoin to my address (but it hadn't yet confirmed), so I'm pretty sure AdamISZ is right

[AdamISZ]

Ah, so it was immediately after sending 1btc to the wallet? Hmm that's slightly different from what I was thinking, I thought in that case it would immediately quit during the startup (not able to generate an offer). Will take a look at that as well, that isn't supposed to happen either :) (The description above should be a very rare case, where you filled an amount almost exactly equal to what you funded with).

[AdamISZ]

@RHavar i don't suppose you overwrote the default config? By default listunspent_args uses bitcoind default, [1,9999999] which means that the yieldgenerator will only recognize coins with at least 1 confirmation (note: wallet-tool.py will, by default, recognize unconfirmed; we set that specifically so a casual user will not be confused), and if your pay-in was unconfirmed you should have seen a quit with the error message 'do not have any coins left', not a crash like the one you posted. If on the other hand listunspent_args was set to [0], the crash you showed seems likely to happen.

[RHavar]

The only changes I made the default config was to set:

cjfee_a = 200
cjfee_r = '0.000'
ordertype = 'absoffer'

[AdamISZ]

Right, that's in the .py script itself, I meant joinmarket.cfg. So since you didn't change it, I'm not sure what could have caused it. A scrubbed log could indeed help, although this isn't a huge deal for you, but we'd really like to know what caused it (even if it only means the difference between a helpful 'not enough coins' message and a crash).

[RHavar]

Not sure which log you need, so just uploaded them all: https://github.com/RHavar/joinmarket/tree/master/logs

[AdamISZ]

OK, so from reviewing the logs here's the situation:

Maker calls create_my_orders on startup, shows 'do not have any coins left' error message but does not shut down, as it used to. This bug must have been introduced during the refactoring for 0.2. It'll be a simple matter to fix that.

For the other logs, things seem to be fine except at one point you lost the connection to CgAn, but as intended, the bot continued to operate fine on agora (including several transactions).

At some point I or someone else should make a PR to fix that: (a) shut down in no coins condition on startup, (b)do not attempt to announce orders if orderlist is empty. Should be pretty simple to do.