search

askmike / gekko

A bitcoin trading bot written in node - https://gekko.wizb.it/
MIT License
10.05k stars 3.95k forks source link

Failed to load historical trades from cex.io #72

Closed rmtcntrl1 closed 8 years ago

rmtcntrl1 commented 10 years ago

When I start the bot, It works correctly until this point:

2013-12-08 19:09:44 (INFO):     Calculating EMA on historical data...
2013-12-08 19:09:44 (DEBUG):    fetching historical data at cex.io
2013-12-08 19:09:44 (DEBUG):    cex.io fetching from tid 670616
2013-12-08 19:09:45 (DEBUG):    fetching since 1386166184

C:\Users\USERFOLDER\Desktop\gekko-master\methods\realtime-candle-fetcher.js:98
    throw 'Failed to load historical trades from ' + this.watcher.name;
                                                   ^
Failed to load historical trades from cex.io

Am I missing something in the config.js? Or cex.io changed his APIs?

n8twj commented 10 years ago

I am seeing the same thing (just cloned and installed gekko today)

obscurecloud commented 10 years ago

I'm getting issues with cex as well

with this in the config:

config.normal = {
  enabled: true,
  exchange: 'cexio', // 'MtGox', 'BTCe', 'Bitstamp' or 'cexio'
  currency: 'BTC',
  asset: 'GHS',
  tradingEnabled: false,
  key: 'xxx',
  secret: 'xxx',
  username: "xxx" // your username, only fill in when using bitstamp or cexio
}

I get this when starting gekko:

2013-12-09 02:32:15 (INFO): I'm gonna make you rich, Bud Fox.
2013-12-09 02:32:15 (INFO): Let me show you some Exponential Moving Averages.

2013-12-09 02:32:15 (INFO): Using normal settings to monitor the live market
2013-12-09 02:32:15 (INFO): NOT trading with real money
2013-12-09 02:32:15 (INFO): Profit reporter active on simulated balance
2013-12-09 02:32:15 (INFO): Calculating EMA on historical data...
2013-12-09 02:32:15 (DEBUG):    fetching historical data at cex.io
2013-12-09 02:32:15 (DEBUG):    cex.io fetching from tid 693213
2013-12-09 02:32:19 (DEBUG):    fetching since 1386221535

/Volumes/HD/Sites/Test/node/gekko/methods/realtime-candle-fetcher.js:98
    throw 'Failed to load historical trades from ' + this.watcher.name;
                                                   ^
Failed to load historical trades from cex.io

But when I activate the normal:

config.normal = {
  enabled: true,
  exchange: 'cexio', // 'MtGox', 'BTCe', 'Bitstamp' or 'cexio'
  currency: 'BTC',
  asset: 'GHS',
  tradingEnabled: true,
  key: 'xxx',
  secret: 'xxx',
  username: "xxx" // your username, only fill in when using bitstamp or cexio
}

I get this:

2013-12-09 02:31:12 (INFO): I'm gonna make you rich, Bud Fox.
2013-12-09 02:31:12 (INFO): Let me show you some Exponential Moving Averages.

2013-12-09 02:31:12 (INFO): Using normal settings to monitor the live market
2013-12-09 02:31:12 (INFO): Profit reporter active on simulated balance
2013-12-09 02:31:12 (DEBUG):    getting balance & fee from cex.io

/Volumes/HD/Sites/Test/node/gekko/exchanges/cexio.js:168
    assets = parseFloat(data.GHS.available) - parseFloat(data.GHS.orders);
                                ^
TypeError: Cannot read property 'available' of undefined
    at calculate (/Volumes/HD/Sites/Test/node/gekko/exchanges/cexio.js:168:33)
    at bound (/Volumes/HD/Sites/Test/node/gekko/node_modules/lodash/dist/lodash.js:729:21)
    at IncomingMessage.<anonymous> (/Volumes/HD/Sites/Test/node/gekko/node_modules/cexio/cexio.js:63:7)
    at IncomingMessage.EventEmitter.emit (events.js:117:20)
    at _stream_readable.js:920:16
    at process._tickCallback (node.js:415:13)

I'm on a mac, installed node with brew, fresh gekko install from today, but haven't tried restarting my computer :)

dylanzr commented 10 years ago

I have the same issue on a fresh Ubuntu 13.10 with nodejs v0.10.15.

GoodDragon commented 10 years ago

I can see that there is lot of problems with gex.io. That why I'm using btc-e.

Anyway you maybe should look these ones #61 and https://github.com/toomyem/gekko/commit/4ec133941106ff4e493bbee650c295c2ac15578c

silverbucket commented 10 years ago

+1 getting this issue as well.

brampower commented 10 years ago

Same here, I guess the cex.io file is bugged for now. Hope to see an update soon. Amazing work!

streetlogics commented 10 years ago

Hey - I'm not a node.js pro at all, barely used it to be honest, but I've done a lot of javascript. I was just playing around with the node debugger while trying to start gekko with cex.io, and I noticed that it was dying at this line essentially: https://github.com/askmike/gekko/blob/master/methods/realtime-candle-fetcher.js#L67 - while looking at the way that code is layed out, it doesn't seem like it's actually even possible for emptyBucket to ever be set to anything but true. I added some debugger breakpoints and played around with the data, and there is definitely "this.currentBucket" data available, and there's also data in cexio.db, so it doesn't make sense that it would throw an error saying there wasn't any data available.

However - when I changed that to "break;" instead of "return false;", it seemed to just get stuck after saying "Calculating EMA on historical data…", unless it was just waiting for something to happen in the market to give me advice

Thoughts? (again, feel free to tell me I have no idea what I'm talking about and to keep walking….) :)

streetlogics commented 10 years ago

Call me crazy, but I think that definitely fixed it, because i turned on debug settings and saw a lot of nice output with the last line showing: "we are currently not in an up or down trend @ 0.06939383 (-0.014)" (that's just according to whatever the default settings are for config.EMA, probably need to be adjusted for CEX standards).

brampower commented 10 years ago

Streetlogics, that's awesome! It works for me, too. Thank you!

This is my output: 2013-12-10 18:51:36 (DEBUG): we are currently in a downtrend @ 0.0694 (-0.582) 2013-12-10 18:51:36 (INFO): ADVICE is to SELL @ 0.0694 (-0.582)

That means it's working, right? I don't know why it's advice is to sell, though. Prices are usually above .07, so it would make no sense to sell at .069.

It is throwing a bunch of these errors at me, though: 2013-12-10 18:51:36 (DEBUG): WARNING: Unable to calculate EMA on current cand le. Using last defined value.

streetlogics commented 10 years ago

Yah I saw those warnings too, but it still looked like it was calculating EMA's. I tweaked the buy and sell thresholds down a bit: sellTreshold: -0.0025, buyTreshold: 0.0025

and now get: 2013-12-10 17:26:35 (INFO): ADVICE is to BUY @ 0.06999911 (0.096)

brampower commented 10 years ago

What interval are you using? The default 60? I just set it to 5 and now it's saying there is no trend (which I don't think there is since it has been around .069 for a while)

And do you know what changing the thresholds does? I think .025 isn't bad; it's what a lot of people use. I just don't understand why it would tell us to sell, though.

streetlogics commented 10 years ago

I changed it to 1 minute to get new prices often since I'm really working with micro trades on cex to start to see how it goes (only gave it .1btc to play with, but have trading disabled for a bit to see what projected profits would be against the live data)

brampower commented 10 years ago

Yeah, for some reason my profit calculator never shows up? I have it set to enabled in the config, but I never see it. Not sure how to know whether it would make me a profit or not.

obscurecloud commented 10 years ago

i made the change listed above, and that will get the node running, but it still fails if i set it to enable trades.

anyone get this working on CEX yet? if so, how please?

rmtcntrl commented 10 years ago

Thank you Streetlogics! It works for me! Although I'm not sure that's the right solution. As brampower said above, it returns a bunch of errors and warning.

@obscurecloud Are you sure that your config.js is properly configurated?

brampower commented 10 years ago

Bwdolphin, enabling verbose did the trick. Thanks!

What are the tresholds you have been using? I can't seem to find the sweet spot.

streetlogics commented 10 years ago

(reposting from the correct account): Yah I'm not sure it's right solution or not either, although I only see warnings, no actual "errors" that keep things from running, so I would assume a "warning" is by design, but I could be wrong.

As for live trading, I did enable live trading for a bit with mine and things worked just fine (had CEX open and saw buy and sell orders go through). Are you sure you entered all your credentials correctly, including your CEX username? You also have to activate the API by clicking an email confirmation link.

Brampower - I had to make sure CEX was set to enabled:true in the "watcher" settings before I saw the profit calc show up. I also ended up enabling verbose output and turned on debug output and was able to see profit calculations

streetlogics commented 10 years ago

bram - i tested a bunch of different thresholds the other day and they all ended in a loss, so I haven't found anything useful/worthwhile yet for CEX.

brampower commented 10 years ago

Interesting. I seriously wonder if this program could work for CEX.io. For some reason it always tells me to buy at very strange times. Price is .073 right now (quite high), and it's saying to buy.

streetlogics commented 10 years ago

Yah - I'm still not convinced my "fix" actually fixed anything and didn't just make things worse hah! :) Who knows, I'm just having fun playing around with nodejs a bit. I wish askmike would swoop into this discussion and save us with some direction!!! :)

GoodDragon commented 10 years ago

@brampower if you have example default setting "buyTreshold: 0.25" and EMA is now .073 then it should buy. That is how it is planned to working because EMA .073 is meaning that price is going up.

There is lot of more information about that. Look following:

75

https://github.com/askmike/gekko/blob/master/docs/Backtesting.md

25

https://github.com/askmike/gekko/blob/master/methods/exponential-moving-averages.js

Anyway, probably some other method instead of EMA will work better. Some one just need create method script for that :)

streetlogics commented 10 years ago

For reference, he was saying the current price of GH in BTC in .073, not the current EMA, but good info none the less

brampower commented 10 years ago

GoodDragon;

Thanks for your reply. I know that it's above the treshold, but when I said .073 I meant the price was .073 and not the treshold.

.025 seems to be working fairly well for BTC to USD exchanges like BTCe, but Cex.io definitely has problems with any treshold right now :(

GoodDragon commented 10 years ago

Hmm. I did new installation with https://github.com/streetlogics/gekko/commit/5f7532403017c84937f25e2e3426622fb072bcb0 fix.

Then running simulation using following settings (verbose and debug also enabled) interval: 5, short: 1, long: 5, sellTreshold: -0.025, buyTreshold: 0.025,

And got results like this: 2013-12-12 21:07:10 (DEBUG): we are currently in a downtrend @ 0.07365001 (-0.039) 2013-12-12 21:07:10 (INFO): ADVICE is to SELL @ 0.07365001 (-0.039)

2013-12-12 21:11:48 (DEBUG): we are currently not in an up or down trend @ 0.07369002 (0.010) 2013-12-12 21:11:48 (INFO): ADVICE is to HOLD @ 0.07369002 (0.010)

2013-12-12 21:16:49 (DEBUG): we are currently not in an up or down trend @ 0.07370996 (0.025) 2013-12-12 21:16:49 (INFO): ADVICE is to HOLD @ 0.07370996 (0.025)

2013-12-12 21:21:48 (DEBUG): we are currently not in an up or down trend @ 0.07370001 (0.007) 2013-12-12 21:21:48 (INFO): ADVICE is to HOLD @ 0.07370001 (0.007)

So I would say that it is working OK in simulation mode and that fix should be commited.

n8twj commented 10 years ago

Is interval how often the API is queried? cex.io/api says:

"Do not make more than 600 request per 10 minutes or we will ban your IP address."

On Thu, Dec 12, 2013 at 12:07 PM, GoodDragon notifications@github.comwrote:

Hmm. I did new installation with streetlogics@5f75324https://github.com/streetlogics/gekko/commit/5f7532403017c84937f25e2e3426622fb072bcb0fix.

Then running simulation using following settings (verbose and debug also enabled) interval: 5, short: 1, long: 5, sellTreshold: -0.025, buyTreshold: 0.025,

And got results like this: 2013-12-12 21:07:10 (DEBUG): we are currently in a downtrend @ 0.07365001 (-0.039) 2013-12-12 21:07:10 (INFO): ADVICE is to SELL @ 0.07365001 (-0.039)

2013-12-12 21:11:48 (DEBUG): we are currently not in an up or down trend @ 0.07369002 (0.010) 2013-12-12 21:11:48 (INFO): ADVICE is to HOLD @ 0.07369002 (0.010)

2013-12-12 21:16:49 (DEBUG): we are currently not in an up or down trend @ 0.07370996 (0.025) 2013-12-12 21:16:49 (INFO): ADVICE is to HOLD @ 0.07370996 (0.025)

2013-12-12 21:21:48 (DEBUG): we are currently not in an up or down trend @ 0.07370001 (0.007) 2013-12-12 21:21:48 (INFO): ADVICE is to HOLD @ 0.07370001 (0.007)

So I would say that it is working OK in simulation mode and that fix should be commited.

— Reply to this email directly or view it on GitHubhttps://github.com/askmike/gekko/issues/72#issuecomment-30456944 .

GoodDragon commented 10 years ago

If I understand code right. It will do only one request per interval to get old trades and then maybe another if it want buy or sell.

600 request per 10 minutes mean that you can do one request per second?

paraghardas commented 10 years ago

For cex.io ... as the price movement is not sufficient ... ema crossover wont work. Also as there is no fees on cex.io for trading ... grid trading will work best ... i.e ... go on buying as price go down at regular interval and go on selling as the price go up ... can such a grid be programmed? ... may be on another project? anyone?

silverbucket commented 10 years ago

I would be willing to help with the coding, etc., but we'd need a sound algorithm/set of rules which I'm not much use for.

On Thu, Dec 19, 2013 at 10:11 AM, paraghardas notifications@github.comwrote:

For cex.io ... as the price movement is not sufficient ... ema crossover wont work. Also as there is no fees on cex.io for trading ... grid trading will work best ... i.e ... go on buying as price go down at regular interval and go on selling as the price go up ... can such a grid be programmed? ... may be on another project? anyone?

— Reply to this email directly or view it on GitHubhttps://github.com/askmike/gekko/issues/72#issuecomment-30914418 .

paraghardas commented 10 years ago

silverbucket - are you referring to my strategy?

silverbucket commented 10 years ago

yup!

On Thu, Dec 19, 2013 at 10:59 AM, paraghardas notifications@github.comwrote:

silverbucket - are you referring to my strategy?

— Reply to this email directly or view it on GitHubhttps://github.com/askmike/gekko/issues/72#issuecomment-30917070 .

paraghardas commented 10 years ago

Please tell me how we proceed ... I will explain what exactly should be done.

silverbucket commented 10 years ago

send me an email, you can find it on my home page (linked to from my github profile)

On Thu, Dec 19, 2013 at 11:14 AM, paraghardas notifications@github.comwrote:

Please tell me how we proceed ... I will explain what exactly should be done.

— Reply to this email directly or view it on GitHubhttps://github.com/askmike/gekko/issues/72#issuecomment-30918004 .

Devouring commented 10 years ago

@silverbucket @paraghardas no need to create a new project you just need to add a new trading method instead of the current one.

streetlogics commented 10 years ago

https://github.com/askmike/gekko/pull/85

streetlogics commented 10 years ago

@silverbucket @paraghardas @Devouring - I went ahead and created a new, separate ticket for the new trading method for CEX - https://github.com/askmike/gekko/issues/86

askmike commented 10 years ago

Right now it is not possible to use CEX.io unless you've build a historical database from before the different API behaviour (= anyone who is already running gekko at CEX.io succesfully).

See this for more info.

askmike commented 10 years ago

A new version of Gekko (in development, need help testing) that holds a local database of all historical data. If you guys can help me test / figure it out we can have this version as soon as possible.

More info here.