realtime gekko doesn't properly handle OS standby

[askmike]

A realtime gekko will poll an exchange regularly for market data and then convert the data into candles. However, if your system goes into standby mode whilst running Gekko, Gekko assumes no trades happened at the exchange and will calculate incorrect candles:

[elGuille]

I am still receiving this error all the time I launch a live trading gekko on Mac OS. Any idea?

[askmike]

@elGuille what kind of computer do you use? If you for example use a macbook and it goes standby in any way (you close the laptop, you don't touch it for a few hours with default settings), it suspends all applications including Gekko. Gekko needs to run non stop so it can keep monitoring the market in realtime.

[elGuille]

@askmike thank you for your quick reply! I have a Macbook Pro and I set the EnergySaver setting to never sleep and never put the hard drive to sleep either.

[askmike]

Strange, it works great on my macbook pro for hours. After how long does this usually happen? Also on what exchange?

[elGuille]

Less than 5 minutes, Im trying Kraken

[askmike]

This might actually be related to kraken, their API is having some trouble at the moment. Could you check whether other exchanges (poloniex for example) work? A paper trader should be sufficient, with regards to this message there is no difference with a real trader and a paper trader.

On Mon, Nov 20, 2017 at 12:24 PM, Guille notifications@github.com wrote:

Less than 5 minutes, Im trying Kraken

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/askmike/gekko/issues/514#issuecomment-345594907, or mute the thread https://github.com/notifications/unsubscribe-auth/AA7MD6_Gn4JaecLdGfg6dWqCFcLjKD8oks5s4Q0fgaJpZM4LDq4K .

-- PGP key at keybase.io/mikevanrossum https://keybase.io/mikevanrossum/key.asc

[elGuille]

I will do that, thank you!

[askmike]

those look good, but you can disable the: margin trading and margin funding one as well.

[elGuille]

Thanks a lot!

[arnoldski]

Experiencing the same problem: "ERROR: Failed to tick in time, see https://github.com/askmike/gekko/issues/514 for details".

Using "binance" as exchange.

[gugol2]

Happened to me with binance and macOS

[antonrifco]

happened with binance

[vuotian]

Happened also in Azure VM, should not go to sleep. Exchange is Bitfinex

[IDK1973]

Hello,

I just got my GEKKO to work and set up some customize strategies. When running real time trade I get ERROR: Failed to tick #514 in time after some about 10 to 20 minutes. I am on GDAX and using windows 10. Is there a fix for this?

[iquirino]

I'm having this issue in an dedicated server under google cloud platform.

Using Poloniex.

2vCPU + 3GB RAM Ubuntu 16.04 LTS Dedicated Instalation Sqlite

Why child process has died?

https://i.imgur.com/w68utcL.png

[bennycode]

For me this happened:

2018-02-10 16:19:37 (DEBUG):    Requested LTC/BNB trade data from Binance ...
Unhandled rejection TimeoutError: operation timed out
    at afterTimeout (D:\dev\projects\github\gekko\node_modules\node-telegram-bot-api\node_modules\request-promise\node_modules\bluebird\js\main\timers.js:16:15)
    at Timeout.timeoutTimeout [as _onTimeout] (D:\dev\projects\github\gekko\node_modules\node-telegram-bot-api\node_modules\request-promise\node_modules\bluebird\js\main\timers.js:59:9)
    at ontimeout (timers.js:475:11)
    at tryOnTimeout (timers.js:310:5)
    at Timer.listOnTimeout (timers.js:270:5)
2018-02-10 16:20:21 (DEBUG):    Processing 2 new trades. From 2018-02-10 15:19:34 UTC to 2018-02-10 15:19:57 UTC. (a few seconds)

 ERROR: Failed to tick in time, see https://github.com/askmike/gekko/issues/514 for details

Operating System: Windows 10 Exchange: Binance

Is it a good advice to start gekko with tools like forever to recover from this scenario?

[donkykong017]

Just FYI. This just happend to me an an AWS EC2. Seems strange that that will ever go to sleepmode. tried to connect to my live instance running for about a week now and then it crashed. market watchers all disappeared and the error was in the command output. restarted watchers but on bitstamp it misses some candles now.

[askmike]

There might have been an error in the connection between gekko and the exchange.

[donkykong017]

thx @askmike for the reply. yeah could be. but i suspect something with the AWS itself or the setup with nginx (other issue i know, but i seem to use unsecure connection although i set it up exactly like in the guide) as all market watchers crashed when i tried to login. gdax and bitstamp

[donkykong017]

i had the same error again. same result. all market watchers crashed. i did try to see my available datasets in local data, but they loaded and maybe got stuck. maybe it has something to do with this limitation as i have quite a lot of datasets: https://github.com/askmike/gekko/commit/7cf20f6900546ffa128d55091971134693204bf5

have to update and see if it helps

[CmvrkDunn]

Has there been any update or resolution to this issue? I just had this occur on a clean Ubuntu vm running locally in vmware workstation 14 player on my desktop that is not set to power off or suspend state. Currently on kraken..

[askmike]

@CmvrkDunn this issue is meant to describe why Gekko stopped, it's not an issue with Gekko but with your system not letting Gekko run non stop. Does it also happen outside the vm (or on any other system)?

[CmvrkDunn]

Sorry, I didn't mean to imply it was an issue with gekko specifically. Has there been general settings identified that need to be set in an OS so it doesn't disrupt gekko? I have had this happen twice within the VM which was running for about a day before presenting the error. I have not experienced it outside the VM but to be fair I haven't run it long outside. Not sure if it has any bearing on this issue but I did have my kraken tickrate at 5, so just moved back to the default of 20 as in the config comments and will update if this resolves my issue. Any thoughts?

[dumbanddumbererest]

happens to my macbook air even with caffeinate running. Doesn't happen if I keep the display open and on mains power.

i have a pretty good strat (non-TA) and was testing it live to see if i should put money into a raspberry pi to run gekko headless. Was on binance exchange. Happened as expected when went to sleep on battery mode, however with caffienate running on mains power it was unexpected.

Probably not relevant to pi anyway

[Jabbaxx]

Hello,

I had NO STANDBY, but anyway had this problem!

I am running a Amazon EC2 ubuntu 16.04 server. It's running 24/7. Just had this happening 3 times in the last 24 hours. All my 4 productive strategies restarted + all paper traders + all candle writers also.

Can it have something to do with time changing ?

There is no NTP running, but maybe the amazon host server "decided" to change the time as it was a bit off. Have in my work often had problems for instance on Hyper-V/VMware with Active Directory servers that did not like that time changes suddently. My strats are running in a bash script that loops and restarts it. The bash script waits for xx:x0:00 before it starts gekko. When the strategy crashes it there is one minute or more between that the last trades was outputted and the strategy is restarted. Here 01:46:33 it outputted last trades and 01:47:49 it gets to start the strategy again. This is almost 1,5 minutes.

2018-05-06 01:46:33 (DEBUG):    Requested BTC/USDT trade data from Binance ...

 ERROR: Failed to tick in time, see https://github.com/askmike/gekko/issues/514 for details

7:49 = waiting for 0:00

Or is gekko just trying so long time before giving up ? Please enlighten me what is happening, so I can help fixing this :)

[Jabbaxx]

It continued all the time and I had to reboot the server to get it to stop :( We should dig deeper in this. Please tell me what I can do to debug it next time ?

[Jabbaxx]

@askmike

Suggested change (not 100% fix)

I have this problem on Amazon EC2 servers. It looks like they sometime are very slow, so alomost nothing happens for a minute and then gekko crashes. I understand that if this happens for an hour or a day, that is bad. But right now the max period is only 1 minute and that is too short. My suggestion is that we take candlesize as a maximum. It could also be 50% of candlesize, but for now I go with candlesize. I have changed heart.js in my systems to be like this:

Heart.prototype.tick = function() { if(this.lastTick) { var CandleSize = 1; if (util.getConfig().tradingAdvisor.candleSize) CandleSize = util.getConfig().tradingAdvisor.candleSize; // make sure the last tick happened within the last candle // @link https://github.com/askmike/gekko/issues/514 if(this.lastTick < moment().unix() - TICKRATE 3) log.debug('tick in time, see https://github.com/askmike/gekko/issues/514 for details.'); if(this.lastTick < moment().unix() - CandleSize 60) util.die('Failed to tick in time, see https://github.com/askmike/gekko/issues/514 for details', true); }

that gives the following error on a test system with 10 min candles, that I just put to sleep for 3 minutes, but otherwise it looks as if it works OK. I will test it now in production and see how it goes :) :

2018-05-11 08:47:28 (DEBUG): Requested XBT/EUR trade data from Kraken ... 2018-05-11 08:47:29 (DEBUG): Processing 5 new trades. From 2018-05-11 05:47:24 UTC to 2018-05-11 05:47:24 UTC. (a few seconds) 2018-05-11 08:50:05 (DEBUG): tick in time, see https://github.com/askmike/gekko/issues/514 for details. 2018-05-11 08:50:05 (DEBUG): Requested XBT/EUR trade data from Kraken ... Unhandled rejection TimeoutError: operation timed out at afterTimeout (C:\Trade\gekko-dev-0.5.13\node_modules\node-telegram-bot-api\node_modules\request-promise\node_modules\bluebird\js\main\timers.js:16:15) at Timeout.timeoutTimeout [as _onTimeout] (C:\Trade\gekko-dev-0.5.13\node_modules\node-telegram-bot-api\node_modules\request-promise\node_modules\bluebird\js\main\timers.js:59:9) at ontimeout (timers.js:475:11) at tryOnTimeout (timers.js:310:5) at Timer.listOnTimeout (timers.js:270:5) 2018-05-11 08:50:15 (DEBUG): Processing 39 new trades. From 2018-05-11 05:47:49 UTC to 2018-05-11 05:50:06 UTC. (2 minutes) 2018-05-11 08:50:25 (DEBUG): Requested XBT/EUR trade data from Kraken ... 2018-05-11 08:50:26 (DEBUG): Processing 4 new trades. From 2018-05-11 05:50:19 UTC to 2018-05-11 05:50:24 UTC. (a few seconds) 2018-05-11 08:50:45 (DEBUG): Requested XBT/EUR trade data from Kraken ... 2018-05-11 08:50:46 (DEBUG): Processing 4 new trades. From 2018-05-11 05:50:35 UTC to 2018-05-11 05:50:45 UTC. (a few seconds)

[shakisha]

happened to me today

[philipina]

Hello,

I'm new in using Gekko. I got exactly the same problem. However, in my case I did many tests with different currency peer and identify that it seems that we get this "Failed to tick in time," error if I monitor a pair with low trading volume. For example, if I monitor BNBBTC every minute it works fine, but if I try to monitor LUNBTC it does not work properly. I think that this is due to the fact that for those low trading volume pair it can be that there is no trade at all during few minutes (e.g. 10 minutes). I have the feeling that in this case, if there is no price move and if there is no trade for a while, Gekko generates this "Failed to tick in time," error.

Let me know if you need more explanations about my case.

[archi44]

There was such a problem Win 7 VPS server. During the paper trade 9 o'clock the robot warmed up. Then he started a paper trade and after some time an error appeared. I tried again and again and every time this error, the exchange Bitfinex.