askmike / gekko

A bitcoin trading bot written in node - https://gekko.wizb.it/
MIT License
10.07k stars 3.94k forks source link

Backtests via UI starting twice #686

Closed thegamecat closed 7 years ago

thegamecat commented 7 years ago

If I run a long backtest via the UI the UI triggers the backtest a second time while the first is running: <-- POST /api/backtest <-- POST /api/backtest

Master branch. LTC 13 months. 1 minute. My own strat using 2 talib operators with no blockers. Same thing happens with one of Gekko strategies.

What would trigger a second post? I can't see anything obvious in the browser.

askmike commented 7 years ago

Can you check the network tab if your browser if it is actually the browser sending both requests?

thegamecat commented 7 years ago

It's not. The request fails with a nothing returned error.

I've gone back to the cli backtester and that's running twice now....going to do another install.

thegamecat commented 7 years ago

Fresh install does not run the CLI backtester twice. However it does call it a second time when using the UI.

askmike commented 7 years ago

However it does call it a second time when using the UI.

In the terminal or in chrome's network tab? The latter is just Chrome's way of displaying the response if the call took a while.

I need more information:

thegamecat commented 7 years ago

I'm running this in chrome incognito to ensure it's not an addon causing it.

There is no additional call in Chrome -> inspect -> network

In console (I added the start console.log in the server.js: <-- POST /api/backtest Start - 2017:04:14:19:28:17 <-- POST /api/backtest Start - 2017:04:14:19:30:17

Runs for at least 10 minutes.

Update: ran for about 26 minutes.

thegamecat commented 7 years ago

This is strange. I'm running the Macd, and it completes but does not report back to the UI, keeps showing the animated gif.

<-- POST /api/backtest 2017:04:14:20:11:24 { currency: 'BTC', asset: 'LTC', startTime: '2016-01-08 18:29:00', endTime: '2017-04-12 17:05:00', timespan: 'a year', market: 21.985275450621998, balance: 100.00961, profit: 0.0017330000000015389, relativeProfit: 0.001732863502340365, yearlyProfit: '0.00137527', relativeYearlyProfit: '0.00137516', startPrice: 0.007878, endPrice: 0.00961, trades: 0, startBalance: 100.007877, alpha: -21.983542450621997 } --> POST /api/backtest 200 83s 106.46mb

Note: I put a console.log report in the papertrader to ensure I see an output / trade report.

thegamecat commented 7 years ago

The second post always occurs after 120 seconds.

thegamecat commented 7 years ago

Adding the timeout line to the server.js seems to fix it:

const wss = new WebSocketServer({ server: server }); wss._server.timeout = 50000000;

thegamecat commented 7 years ago

So now the long 5 year backtests work with 1 minute candles.

Sadly after reporting the papertrader final report to console we see an error:

{ currency: 'USD', asset: 'BTC', startTime: '2011-08-14 14:14:00', endTime: '2017-02-20 01:32:00', timespan: '6 years', market: 9800.269230769229, balance: 154.36272553636, profit: 153.36272553636, relativeProfit: 15336.272553636, yearlyProfit: '27.76008157', relativeYearlyProfit: '2776.00815719', startPrice: 10.4, endPrice: 1029.628, trades: 107, startBalance: 1, alpha: -9646.906505232868 }

RangeError: Invalid string length at JSON.stringify () at Object.get length [as length] (/Users/freefall/gekkoinstalls/gekko/node_modules/koa/lib/response.js:206:55) at Object.logger (/Users/freefall/gekkoinstalls/gekko/node_modules/koa-logger/index.js:60:31) at logger.next () at onFulfilled (/Users/freefall/gekkoinstalls/gekko/node_modules/co/index.js:65:19) at process._tickCallback (internal/process/next_tick.js:103:7)

thegamecat commented 7 years ago

Seems to be an out of memory, due to the size of the array. Not much to do about that then.

askmike commented 7 years ago

Unfortunately yes. A possible solution would be what we discussed here:

https://github.com/askmike/gekko/issues/632#issuecomment-288176842

thegamecat commented 7 years ago

Yep, that works.

This timeout change has been absolutely brilliant when combining multiple talibs which tend to be quite slow. I can also get results with graphs over 1 year with 1 minute now.

askmike commented 7 years ago

It's really interesting to see how far you are able to push Gekko! If you have any other crazy stories please let me know :)