Galts-Gulch / avarice

Multi-Indicator Python3 OKCoin CN & Intl Trading Bot/Infrastructure
Other
91 stars 29 forks source link

Task exception was never retrieved #5

Closed ghost closed 9 years ago

ghost commented 9 years ago

I've had this happen a couple of times after upgrading and started using the websockets version of avarice instead of the api version.

Here is the error I receive:

Task exception was never retrieved future: <Task finished coro=<RunCommon() done, defined at C:\Avarice\avarice.py: 23> exception=TypeError("Can't convert 'int' object to str implicitly",)> Traceback (most recent call last): File "C:\Python34\lib\asyncio\tasks.py", line 236, in _step result = next(coro) File "C:\Avarice\avarice.py", line 51, in RunCommon trd.TradeFromStrategy() File "C:\Avarice\trader.py", line 77, in TradeFromStrategy TradeAmount = GetTradeAmount('sell') File "C:\Avarice\trader.py", line 19, in GetTradeAmount ta = gu.RoundIfGreaterThan(el.GetTradeAmount('asset'), 3) File "C:\Avarice\exchangelayer.py", line 55, in GetTradeAmount Amount = (gc.Trader.TradeVolume / 100) * GetFree('asset') File "C:\Avarice\exchangelayer.py", line 39, in GetFree okwspriv.userinfo())[-1]['data']['info']['funds']['free'][gc.API.Asset] File "C:\Avarice\okcoin\WebSocketAPI.py", line 69, in userinfo 'parameters':{ 'api_key':'" + self.api_key + "', 'sign':'" + sign + "'} }") TypeError: Can't convert 'int' object to str implicitly

ghost commented 9 years ago

I ran it again overnight, and this time encountered a different error. Candle: 12 | Price: 1407.59 cny | Time: 00:08:41 | Date: 2015/01/31 Task exception was never retrieved future: <Task finished coro=<RunCommon() done, defined at C:\Avarice\avarice.py 23> exception=ZeroDivisionError('float division by zero',)> Traceback (most recent call last): File "C:\Python34\lib\asyncio\tasks.py", line 236, in _step result = next(coro) File "C:\Avarice\avarice.py", line 46, in RunCommon getattr(indicators, indicator).indicator() File "C:\Avarice\indicators.py", line 445, in indicator gc.KDJ.FastKPeriod)) File "C:\Avarice\indicators.py", line 62, in FastStochK

RealJohnGalt commented 9 years ago

First of all, thank you for testing. The v2.0 tag is the last non-beta which is functional without the websocket API and you can run it as another instance for now ( https://github.com/Galts-Gulch/avarice/releases/tag/v2.0 ) . I'll also add a note to README about this.

That last error is something I will be pushing a fix for today. I've been testing a workaround for it (the last price is always the same if OKCoin drops the connection partially, but our library think the connection has stayed open). This never popped up prior because we would never get so many prices which were the exact same.

However, I haven't seen that first error and would really like to get that figured out.

  1. Has it ever successfully completed a trade since upgrading?
  2. Do you have '' surrounding both the apikey and secretkey in genconfig? It would look something like this: apikey = '8bdc7c8f-736e-11e4-acdf-00463e0238cc' secretkey = 'D25DA611AB934678AB643DB81D65201A'
  3. At some point, OKCoin changed their key format. I can no longer test this since I don't have the old keys, but I suspect that only the new format is accepted for the websocket API. Could you create a new pair of keys and test those?

Edit: I'll be sure to add a check for the second question as well. I really appreciate this issue.

ghost commented 9 years ago

Thank you for creating an awesome trading bot. I'm an amateur coder and have been working on a way to implement ADX with this bot but I'm not experienced enough to pull it off. :) With that said I'll go through and answer your questions to hopefully help shed some light on this issue.

Has it ever successfully completed a trade since upgrading?

No, I've never gotten it to go more than 35 candles without it generating an error.

Do you have '' surrounding both the apikey and secretkey in genconfig? It would look something like this: apikey = '8bdc7c8f-736e-11e4-acdf-00463e0238cc' secretkey = 'D25DA611AB934678AB643DB81D65201A'

No I have been using the same format since I first started using avarice back when you posted it on reddit. I'll generate a new key pair and use the new format for testing.

RealJohnGalt commented 9 years ago

That would be the issue! I'll be sure to make a note on upgrading. ADX is an awesome request and would work well with how multiple indicators currently work if implemented as a threshold strategy (since all TradingIndicators must have matching signals). Down the road I'll implement another strategy for multiple indicators so each signal is traded on accordingly (for instance you could trade 80% of volume with MACD, and 5% of volume with Stochastic RSI 98/2 without needing to run separate instances).

I'm working out some more of these bugs with the websocket API/asyncio now and should have more of these issues fixed. Long term the websocket API is a far better solution than the REST API. The real issue is how asyncio handles infinite loops (and fake asynchronous loops in general) so the best route might be to use a threadsafe db and fork the function for the public ticker.

As an example of an issue, here are 6 second candles with some added logging:

Connecting to Public OKCoin WebSocket... Getting recent Ticker data Getting recent Ticker data Candle: 1 | Price: 1413.87 cny | Time: 11:22:19 | Date: 2015/01/31 Getting recent Ticker data Candle: 2 | Price: 1413.87 cny | Time: 11:22:25 | Date: 2015/01/31 Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Candle: 3 | Price: 1413.9 cny | Time: 11:22:31 | Date: 2015/01/31 Candle: 4 | Price: 1413.9 cny | Time: 11:22:37 | Date: 2015/01/31 Candle: 5 | Price: 1413.9 cny | Time: 11:22:43 | Date: 2015/01/31 Candle: 6 | Price: 1413.9 cny | Time: 11:22:49 | Date: 2015/01/31 Candle: 7 | Price: 1413.9 cny | Time: 11:22:55 | Date: 2015/01/31 Candle: 8 | Price: 1413.9 cny | Time: 11:23:01 | Date: 2015/01/31 Candle: 9 | Price: 1413.9 cny | Time: 11:23:07 | Date: 2015/01/31 Candle: 10 | Price: 1413.9 cny | Time: 11:23:13 | Date: 2015/01/31 Candle: 11 | Price: 1413.9 cny | Time: 11:23:19 | Date: 2015/01/31

ghost commented 9 years ago

I had never heard of websocket until you implemented it with Avarice. After doing some reading up on that protocol I definitely agree that it is the way to go long term. Full duplex between client and host is a huge advantage over REST API. What's a little confusing is the Public / Private websocket connections, is it 2 separate connections in case one doesn't respond it can fall back to the other? The problem with the current OKCoin API is the damn thing always takes too long to respond. I've even considered renting out some server space close to the exchange to cut down on latency too and from them, but that's a whole other discussion..

I was using this to try and port it over to Avarice, the python ADX indicator is at the bottom of the page. http://sentdex.com/sentiment-analysisbig-data-and-python-tutorials-algorithmic-trading/python-average-directional-index-directional-movement-system-calculation/

Someone has already taken the trouble of coding ADX into python. So maybe that will help cut down some time.

The candles you posted is exactly what Avarice was doing lastnight before it generated that second error. It got to Candle 12, with the same price.

RealJohnGalt commented 9 years ago

They're two separate connections with separate libraries. The reasoning is that when we subscribe to ticker info, we get a consistent stream of trades until we cancel the subscription. All private websocket sends (trade, cancelorder, and userinfo) only reply back once after the send for either info (e.g. userinfo), or a success message.

Off topic regarding servers: one of the lowest cost (but mediocre) options might be to get a system through bithost.io in Singapore. It's not obscenely close, but on a system there I'm able to get ~141.2ms average to okcoin.cn

I've found the issue with the asyncio loop, and should have a fix up very shortly.

ghost commented 9 years ago

Thanks for the explanation on the public / private websocket connection and makes sense now. Once you get the new bot out, I'll run it in simulation and see if I encounter anymore issues.

If I can be of any assistance with the ADX implementation please feel free to let me know.

That's not a bad latency. Here on the east coast in the U.S. I average between 350-400ms response times from the OkCoin.cn servers.

RealJohnGalt commented 9 years ago

That issue's been fixed in f77ea9c and the loop is functioning as it should :+1:

Here's an example with that same logging and 6 second candles (I tested quite a bit longer than 24 seconds):

Connecting to Public OKCoin WebSocket... Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Candle: 1 | Price: 1409.64 cny | Time: 12:28:40 | Date: 2015/01/31 Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Candle: 2 | Price: 1409.5 cny | Time: 12:28:46 | Date: 2015/01/31 Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Candle: 3 | Price: 1409.75 cny | Time: 12:28:52 | Date: 2015/01/31 Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data Getting recent Ticker data

Of course when it says "Getting recent Ticker data," it's actually only reading the last received ticker data. One of the huge benefits is that we only need to subscribe for ticker data once, and then read the received (unlike REST API).

I just opened #6 as an enhancement for ADX support as a reminder. I'll be adding some other issues to handle upcoming changes.

Thank you again.

RealJohnGalt commented 9 years ago

A release with this fix has also been tagged as v3.0b4