Automate Command-Line Backtesting?

[imkane]

What I'd love to be able to do is pass in command-line variables (specific to whatever strategy being tested) to the backtest tool, get results, parse them, and save the results in a DB.

Ideally, this way one could run through every permutation of variables for a strategy and figure out which combination works best.

Has anyone done this yet?

[askmike]

This is definitely a very interesting approach. As part of the UI Gekko now exposes a REST api you can use to do this without a lot of overhead (might be easier than using the commandline version and having to parse the output), but note that the API is currently going through many changes and is not really documented that well.

Using this API it's not to hard to build something on top of Gekko that does what you describe, in #767 @thegamecat shows a solution to what you are asking.

---

Though note that I did not have this use case in mind when I was developing Gekko, which means this will be very slow. To quote myself:

The idea of Gekko is that it is a simple starters kit for building TA strats, keeping simplicity is achieved by sacrificing more advanced (and complex) functionality that only a small portion of users would need but makes everything more confusing for everything else. Examples of this are:

• Simple flow based codebase: 90% of the code for importing is the same as for running backtests is the same as running against a live market.
• No possibilities for execution strategies: your strat advices long or short and based on that your portfolio will be reallocated ASAP.
• No data below 1 minute resolution
• etc.

TLDR: the backtesting code was written with readability in mind, and I am pretty sure that with some effort we can get backtester to be at least 10 times faster. But having a fast backtester was never part of the design goals.

That full discussion (as well as my thoughts on how to get a faster backtester) can be found in #747.

[imkane]

Thanks @askmike! I'll take a look at the API and share anything I come up with.

[askmike]

Great, please note that I am working on a very big update. Which will most likely release this weekend, it has a number of changes from the current stable branch. So if you play around, please play around with this branch: #737.

[imkane]

FYI, with this feature/ui-trader branch, I received this error when trying to start the ui:

Error: Cannot find module '../api-keys.js'
    at Function.Module._resolveFilename (module.js:469:15)
    at Function.Module._load (module.js:417:25)
    at Module.require (module.js:497:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (/home/gekko/web/apiKeyManager.js:19:17)
    at Module._compile (module.js:570:32)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3)

I manually created api-keys.js in the root dir and that fixed it.

However, looking at /web/apiKeyManager.js, it looks like the code should attempt to create the file if it doesn't exist?

[askmike]

Yes, that was the idea. Maybe require calls get hoisted or so? I'll try to fix in in a few minutes.

[askmike]

FYI, with this feature/ui-trader branch, I received this error when trying to start the ui:

I am unable to reproduce, for me it creates the file just fine..

[imkane]

Maybe something unique with my linux config, or some permission issue? Not sure ...