Error running

[jimmykl]

I've installed the latest version of gekko, imported some data and can run a backtest on it. But when I try gekkoga I get the following error:

No previous run data, starting from scratch!
Starting GA with epoch populations of 20, running 5 units at a time!
{ StatusCodeError: 500 - "Internal Server Error"
    at new StatusCodeError (/Users/jimmy/Sites/gekko/gekkoga/node_modules/request-promise-core/lib/errors.js:32:15)
    at Request.plumbing.callback (/Users/jimmy/Sites/gekko/gekkoga/node_modules/request-promise-core/lib/plumbing.js:104:33)
    at Request.RP$callback [as _callback] (/Users/jimmy/Sites/gekko/gekkoga/node_modules/request-promise-core/lib/plumbing.js:46:31)
    at Request.self.callback (/Users/jimmy/Sites/gekko/gekkoga/node_modules/request/request.js:186:22)
    at Request.emit (events.js:160:13)
    at Request.<anonymous> (/Users/jimmy/Sites/gekko/gekkoga/node_modules/request/request.js:1163:10)
    at Request.emit (events.js:160:13)
    at IncomingMessage.<anonymous> (/Users/jimmy/Sites/gekko/gekkoga/node_modules/request/request.js:1085:12)
    at Object.onceWrapper (events.js:255:19)
    at IncomingMessage.emit (events.js:165:20)
    at endReadableNT (_stream_readable.js:1101:12)
    at process._tickCallback (internal/process/next_tick.js:152:19)
  name: 'StatusCodeError',
  statusCode: 500,
  message: '500 - "Internal Server Error"',
  error: 'Internal Server Error',
  options:
   { url: 'http://192.168.1.4:3002/api/backtest',
     json: true,
     body: { gekkoConfig: [Object], data: [Object] },
     headers: { 'Content-Type': 'application/json' },
     timeout: 1200000,
     method: 'POST',
     callback: [Function: RP$callback],
     transform: undefined,
     simple: true,
     resolveWithFullResponse: false,
     transform2xxOnly: false },
  response:
   IncomingMessage {
     _readableState:
      ReadableState {
        objectMode: false,
        highWaterMark: 16384,
        buffer: [BufferList],
        length: 0,
        pipes: null,
        pipesCount: 0,
        flowing: true,
        ended: true,
        endEmitted: true,
        reading: false,
        sync: true,
        needReadable: false,
        emittedReadable: false,
        readableListening: false,
        resumeScheduled: false,
        destroyed: false,
        defaultEncoding: 'utf8',
        awaitDrain: 0,
        readingMore: false,
        decoder: null,
        encoding: null },
     readable: false,
     domain: null,
     _events:
      { end: [Array],
        close: [Array],
        data: [Function],
        error: [Function] },
     _eventsCount: 4,
     _maxListeners: undefined,
     socket:
      Socket {
        connecting: false,
        _hadError: false,
        _handle: null,
        _parent: null,
        _host: null,
        _readableState: [ReadableState],
        readable: false,
        domain: null,
        _events: [Object],
        _eventsCount: 9,
        _maxListeners: undefined,
        _writableState: [WritableState],
        writable: false,
        allowHalfOpen: false,
        _bytesDispatched: 958,
        _sockname: null,
        _pendingData: null,
        _pendingEncoding: '',
        server: null,
        _server: null,
        parser: null,
        _httpMessage: [ClientRequest],
        read: [Function],
        _consuming: true,
        _idleTimeout: -1,
        _idleNext: null,
        _idlePrev: null,
        _idleStart: 1009,
        _destroyed: false,
        [Symbol(asyncId)]: 19,
        [Symbol(bytesRead)]: 176,
        [Symbol(asyncId)]: 62,
        [Symbol(triggerAsyncId)]: 34 },
     connection:
      Socket {
        connecting: false,
        _hadError: false,
        _handle: null,
        _parent: null,
        _host: null,
        _readableState: [ReadableState],
        readable: false,
        domain: null,
        _events: [Object],
        _eventsCount: 9,
        _maxListeners: undefined,
        _writableState: [WritableState],
        writable: false,
        allowHalfOpen: false,
        _bytesDispatched: 958,
        _sockname: null,
        _pendingData: null,
        _pendingEncoding: '',
        server: null,
        _server: null,
        parser: null,
        _httpMessage: [ClientRequest],
        read: [Function],
        _consuming: true,
        _idleTimeout: -1,
        _idleNext: null,
        _idlePrev: null,
        _idleStart: 1009,
        _destroyed: false,
        [Symbol(asyncId)]: 19,
        [Symbol(bytesRead)]: 176,
        [Symbol(asyncId)]: 62,
        [Symbol(triggerAsyncId)]: 34 },
     httpVersionMajor: 1,
     httpVersionMinor: 1,
     httpVersion: '1.1',
     complete: true,
     headers:
      { 'content-type': 'text/plain; charset=utf-8',
        'content-length': '21',
        date: 'Fri, 19 Jan 2018 13:20:49 GMT',
        connection: 'close' },
     rawHeaders:
      [ 'Content-Type',
        'text/plain; charset=utf-8',
        'Content-Length',
        '21',
        'Date',
        'Fri, 19 Jan 2018 13:20:49 GMT',
        'Connection',
        'close' ],
     trailers: {},
     rawTrailers: [],
     upgrade: false,
     url: '',
     method: null,
     statusCode: 500,
     statusMessage: 'Internal Server Error',
     client:
      Socket {
        connecting: false,
        _hadError: false,
        _handle: null,
        _parent: null,
        _host: null,
        _readableState: [ReadableState],
        readable: false,
        domain: null,
        _events: [Object],
        _eventsCount: 9,
        _maxListeners: undefined,
        _writableState: [WritableState],
        writable: false,
        allowHalfOpen: false,
        _bytesDispatched: 958,
        _sockname: null,
        _pendingData: null,
        _pendingEncoding: '',
        server: null,
        _server: null,
        parser: null,
        _httpMessage: [ClientRequest],
        read: [Function],
        _consuming: true,
        _idleTimeout: -1,
        _idleNext: null,
        _idlePrev: null,
        _idleStart: 1009,
        _destroyed: false,
        [Symbol(asyncId)]: 19,
        [Symbol(bytesRead)]: 176,
        [Symbol(asyncId)]: 62,
        [Symbol(triggerAsyncId)]: 34 },
     _consuming: true,
     _dumped: false,
     req:
      ClientRequest {
        domain: null,
        _events: [Object],
        _eventsCount: 6,
        _maxListeners: undefined,
        output: [],
        outputEncodings: [],
        outputCallbacks: [],
        outputSize: 0,
        writable: true,
        _last: true,
        upgrading: false,
        chunkedEncoding: false,
        shouldKeepAlive: false,
        useChunkedEncodingByDefault: true,
        sendDate: false,
        _removedConnection: false,
        _removedContLen: false,
        _removedTE: false,
        _contentLength: null,
        _hasBody: true,
        _trailer: '',
        finished: true,
        _headerSent: true,
        socket: [Socket],
        connection: [Socket],
        _header: 'POST /api/backtest HTTP/1.1\r\nContent-Type: application/json\r\nhost: 192.168.1.4:3002\r\naccept: application/json\r\ncontent-length: 805\r\nConnection: close\r\n\r\n',
        _onPendingData: [Function: noopPendingOutput],
        agent: [Agent],
        socketPath: undefined,
        timeout: undefined,
        method: 'POST',
        path: '/api/backtest',
        _ended: true,
        res: [Circular],
        aborted: undefined,
        timeoutCb: [Function: emitTimeout],
        upgradeOrConnect: false,
        parser: null,
        maxHeadersCount: null,
        [Symbol(outHeadersKey)]: [Object] },
     request:
      Request {
        domain: null,
        _events: [Object],
        _eventsCount: 5,
        _maxListeners: undefined,
        body: '{"gekkoConfig":{"watch":{"exchange":"poloniex","currency":"USDT","asset":"BTC"},"paperTrader":{"slippage":0.05,"feeTaker":0.25,"feeMaker":0.15,"feeUsing":"taker","simulationBalance":{"asset":1,"currency":1},"reportRoundtrips":true,"enabled":true},"writer":{"enabled":false,"logpath":""},"tradingAdvisor":{"enabled":true,"method":"custom_macd","candleSize":240,"historySize":60},"trader":{"enabled":false},"backtest":{"daterange":{"from":"2018-01-18 00:00","to":"2018-01-18 23:58"}},"performanceAnalyzer":{"riskFreeReturn":5,"enabled":true},"valid":true,"custom_macd":{"historySize":60,"macd_short":15,"macd_long":21,"macd_signal":12,"macd_up":"10.51","macd_down":"-18.79","candleSize":240}},"data":{"candleProps":["close","start"],"indicatorResults":false,"report":true,"roundtrips":false,"trades":false}}',
        headers: [Object],
        timeout: 1200000,
        method: 'POST',
        readable: true,
        writable: true,
        explicitMethod: true,
        _qs: [Querystring],
        _auth: [Auth],
        _oauth: [OAuth],
        _multipart: [Multipart],
        _redirect: [Redirect],
        _tunnel: [Tunnel],
        _rp_resolve: [Function],
        _rp_reject: [Function],
        _rp_promise: [Promise],
        _rp_callbackOrig: undefined,
        callback: [Function],
        _rp_options: [Object],
        setHeader: [Function],
        hasHeader: [Function],
        getHeader: [Function],
        removeHeader: [Function],
        localAddress: undefined,
        pool: {},
        dests: [],
        __isRequestRequest: true,
        _callback: [Function: RP$callback],
        uri: [Url],
        proxy: null,
        tunnel: false,
        setHost: true,
        originalCookieHeader: undefined,
        _disableCookies: true,
        _jar: undefined,
        port: '3002',
        host: '192.168.1.4',
        path: '/api/backtest',
        _json: true,
        httpModule: [Object],
        agentClass: [Function],
        agent: [Agent],
        _started: true,
        href: 'http://192.168.1.4:3002/api/backtest',
        req: [ClientRequest],
        ntick: true,
        timeoutTimer: null,
        response: [Circular],
        originalHost: '192.168.1.4:3002',
        originalHostHeaderName: 'host',
        responseContent: [Circular],
        _destdata: true,
        _ended: true,
        _callbackCalled: true },
     toJSON: [Function: responseToJSON],
     caseless: Caseless { dict: [Object] },
     read: [Function],
     body: 'Internal Server Error' } }

[sockRa]

I've got the same issue, also the latest gekko installed. Cloned gekkoga into gekko folder and followed the instructions.

[kazi308]

Exact same issue here. Looks like its an issue on the gekko server side.

[BenVlodgi]

same

[lumyus]

same

[sockRa]

I managed to solve the issue. I don't know why this works but I tried to change to a custom strat and all of the sudden the program began working. I've tried all kinds of strats that gekko comes with but none of them worked. You should try to do the same, you can find custom strats on github if you search for gekko. You also need to import data from gekko ui and change the date in "your-config" in the gekkoga folder, just so you know.

[StephBMG]

same for me stock strategies don't work Supertrend strategy is woking https://github.com/DustinJSilk/gekko-stochastic-strategy#gekko-stochastic-strategy is not working also

[ehartford]

same error

[ehartford]

@sockRa can you please give an example?

[ehartford]

@StephBMG can you please show example of the config that works for supertrend?

[StephBMG]

Supertrend.zip

[padioca]

Has anyone been able to get this to work? I've tried multiple versions of Gekko to no avail. Also tried Supertrend strategy and can't get anything to run, just getting 500 errors.

[padioca]

Finally got it to run. Took the variables at the bottom that were referencing macd out and configured them to use the appropriate settings from whichever strategy I was analyzing and then had it use randomExt to provide variations of the value. However, once that was setup and operating I can't get it to execute any trades regardless of the strategy used. I set the values exactly as they were set in Gekko but Gekko GA would return a different value. Not sure how to troubleshoot that, but I've tried RSI, MACD and DEMA thus far, all with different datasets to no avail.

[ehartford]

thank you for the info, I'll give it a try!

[ehartford]

I got it to work!

I also had to increase the timeout in my gekko/web/vue/UIconfig.js

And I had to run this in my gekko directory:

npm install tulind

[padioca]

Nice! Have you been able to get it to execute any trades or optimize profits? All I can get it to do is run but then it doesn't seem to actually store anything that it is evaluating.

[ehartford]

you have to create a directory gekkoga/results/config and then it will be able to store the results there. Once you have the right parameters stored there, you can use them to backtest and then paper trade and when you're ready, a live trader. I think the longer the genetic algorithm runs, the better the resulting parameters will be.

[padioca]

Are you showing that Gekko GA is actually outputting something into gekkoga/results/config and are you confident that it is actually iterating and outputting information from the winners of the epochs? I created the directory and nothing is being output there and I am still experiencing the same results where the global maximums don't change despite the settings being fixed in a position that should outperform the default market numbers loaded in.

[ehartford]

I am getting a .json file in the result directory.

ubuntu@ip-172-31-43-211:~/gekko/gekkoga/results$ cat your-config-USD_ETH.json | jq
{
  "parameters": {
    "historySize": 82,
    "atrEma": 6,
    "bandFactor": 9,
    "candleSize": 40
  },
  "score": 1070.9984097852177,
  "profit": 686.6123648800001,
  "sharpe": 1.559829773779849,
  "epochNumber": 16,
  "otherMetrics": {
    "balance": 1047.36236488,
    "profit": 686.6123648800001,
    "sharpe": 1.559829773779849,
    "market": 191.31341209173036,
    "relativeProfit": 190.3291378738739,
    "yearlyProfit": "4111.14781442",
    "relativeYearlyProfit": "1139.61131377",
    "startPrice": 359.75,
    "endPrice": 1048,
    "trades": 6
  }
}

Also, to the maintainer I offer 1 ether bounty for this bug to be fixed for all the built-in gekko strategies

[padioca]

I'm seeing the .json file as well. Definitely appears as though it is something pertaining to the built-in in strategies as I am able to get Supertrend.js working

[GryphonR]

Good tips here, thanks,

This seems too simple, but I managed to get the built-in MACD strategy working - the issue there was that the default options for thresholds were in the thresholds array. In the MACD.js strategy I simply removed thresholds from the threshold array and into settings, then updated the strategy to match. eg. settings.threshold.up > settings.up

Then updated the ga config to represent the configuration variables:

short: randomExt.integer(15,5),
long: randomExt.integer(40,15),
signal: randomExt.integer(12,6),
up: randomExt.float(20,0).toFixed(2),
down: randomExt.float(0,-20).toFixed(2),
persistence: randomExt.integer(3,1),

I've not tried this for any other strategies yet.

EDIT:

This seems to work for all built in strategies. GA doesn't seem to like manipulating variables within arrays so removing the arrays from the built in strategies is the easiest way I've found around it

[padioca]

Not trying to be dense here (although I probably am being dense), but I'm not following what you mean. @RJPGriffin - would you mind posting the paths and the altered files that you used to get this working?

[GryphonR]

I haven't actually forked either this or gekko, so can't link to my files but I'll try to improve on my rather poor explanation! It isn't a fix to the issue, just a hacky workaround.

The error that was being thrown by gekko (not gekkoga) when I tried to run the built in strategies through GA was that threshold variables were undefined.

gekko is expecting the settings object to follow this format: (they're called parameters in the GUI, but settings in code)

//RSI Strategy
settings{
    interval : 14,
    thresholds : {
        low : 30,
        high : 70,
        persistence : 1
    }
}

but ga doesn't seem to be able to generate parameters within the thresholds sub-array (I haven't tried very hard at it though). My solution was to copy the built in strategy (in gekko/strategies) I wanted to use and then edit it so that when it references variables that GA is altering, they are all in the first tier of the settings{} object:

//RSI Strategy
settings{
    interval : 14,
    low : 30,
    high : 70,
    persistence : 1
}

To do this, in the gekko strategy you can simply do a find and replace. Find "thresholds." replace with "". Sticking with the RSI Example, this changes:

//Line 66 from
if(this.stochRSI > this.settings.thresholds.high){
//to
if(this.stochRSI > this.settings.high){

//Line 80 from
if(this.trend.duration >= this.settings.thresholds.persistence)
//to
if(this.trend.duration >= this.settings.persistence)

And the same to any references to thresholds.low and thresholds.interval. Hopefully you get the idea.

We now have a built in strategy lightly modified to work with GA.

Moving on to the GA side, I copied the sample config in gekkoga/config/ into a new file. I called it RSI-config.js.

In this file, in the getProperties function (around line 60) we now need to declare the matching properties. For RSI this is:

historySize : randomExt.integer(40,10),
low : randomExt.integer(30,5),
high : randomExt.integer(95,70),
interval : randomExt.integer(3,1),
persistence : randomExt.integer(3,1)

I hope that makes more sense...

[SebFourault]

Thanks @RJPGriffin for explaining the hack 😃 I think it relates to what is explained in the sample-config.js original file (line 61) :

// Strat settings must be flattened and cannot be nested for mutation to work properly!

So as you say, GA won't work with a settings array including sub-arrays, we must create our own custom strategy file with a "1 level-deep array" to match this need !

[rkingy]

Hi All, I also ran into issues with various indicators getting settings they weren't expecting.

Easiest way to figure out where it is going wrong is one by one hard code into the strat the indicator settings until it works.

For example:

  var macdSettings = {};
  macdSettings.optInFastPeriod = this.settings.MACDshort;
  macdSettings.optInSlowPeriod = this.settings.MACDlong;
  macdSettings.optInSignalPeriod = this.settings.MACDsignal;
  this.addTulipIndicator('macd', 'macd', macdSettings);

The MACDshort must always be lower than MACDlong for the Tulip indicator to work. Otherwise it explodes with the 500 error.

Too work around this, I did the following in my gekkoga config.

candleValues: [5,10,15],
macdShort: 12,
getProperties: () => ({
    // Strat settings must be flattened and cannot be nested for mutation to work properly!
        MACDshort: randomExt.integer(config.macdShort, 3),
    MACDlong: randomExt.integer(50, (config.macdShort +1)),
    MACDsignal: randomExt.integer(20, 3),

So MACDlong is always larger than macdShort.

Hope this helps someone, I was sluicing around the internet for a while looking for the best solution.

[generalectric]

@rkingy correct and I believe that is what has given the majority of people a hard time. However gekkoga now supports nested objects, this should no longer be an issue.