search

Gab0 / japonicus

Genetic Algorithm for Gekko Trading Bot.
MIT License
284 stars 103 forks source link

Error: config failure - happens randomly after running Japonicus for a while #152

Open nmikaty opened 6 years ago

nmikaty commented 6 years ago

Hi everyone,

When I run Japonicus for one of my custom strategies it seems to work fine at first, then suddenly at a random point in time it will start giving Error: config failure errors and finally crash.

This can happen at any time, sometimes at epoch 2, sometimes at epoch 50. Below is an example of a crash on locale 11 at epoch 53.

To give some context: I run Japonicus on windows 10. I just updated Gekko (v0.6.4) and Japonicus (updated for new Gekko) and run on 'develop' branches. I use large initial populations (10 locales of 100 individuals) and a large time window (180 days), so the crashs can happen after several hours. The Gekko console doesn't show any error.

Locale10
first unevaluated: 44
17 individues removed due to equality
[27]
EPOCH 53        &27
Maximum profit 172.903     Average profit 67.052
Minimum profit -281.097    Profit variation 133.652
Population size 89         Max population size 99
Avg trade number 311.037   Avg sharpe ratio 159.199
Avg exposure time 25.314

Locale11
first unevaluated: 27
10 individues removed due to equality
[17]

Error: config failure
http://localhost:3000/api/backtest
{'watch': {'exchange': 'bitfinex', 'currency': 'USD', 'asset': 'BTC'}, 'paperTrader': {'fee': 0.25, 'feeMaker': 0.15, 'feeTaker': 0.25, 'feeUsing': 'maker', 'slippage': 0.05, 'simulationBalance': {'asset': 0, 'currency': 100}, 'reportRoundtrips': True, 'enabled': True}, 'tradingAdvisor': {'enabled': True, 'method': 'Nico_RBB_ADX2_BB', 'candleSize': 5, 'historySize': 2000}, 'Nico_RBB_ADX2_BB': {'ADX': {'adx': 3.54, 'high': 42.0, 'low': 40.0}, 'BBands': {'NbDevDn': 2.0, 'NbDevUp': 2.0, 'TimePeriod': 20.0}, 'BBtrend': {'bearPersistence': 14.870000000000001, 'bullPersistence': 3.8000000000000003, 'lowerThreshold': 50.8, 'upperThreshold': 73.0}, 'BEAR': {'high': 54.6, 'low': 30.4, 'mod_high': -16.0, 'mod_low': -8.7, 'rsi': 7.8}, 'BULL': {'high': 89.6, 'low': 32.0, 'mod_high': 10.65, 'mod_low': -16.0, 'rsi': 10.04}, 'SMA': {'long': 860.0, 'short': 48.0}}, 'backtest': {'daterange': {'from': '2018-01-31 18:57:09', 'to': '2018-07-30 18:57:09'}}, 'performanceAnalyzer': {'riskFreeReturn': 2, 'enabled': True}, 'valid': True, 'data': {'candleProps': ['id', 'start', 'open', 'high', 'low', 'close', 'vwp', 'volume', 'trades'], 'indicatorResults': True, 'report': True, 'roundtrips': True, 'trades': True}, 'backtestResultExporter': {'enabled': True, 'writeToDisk': False, 'data': {'stratUpdates': False, 'roundtrips': True, 'stratCandles': False, 'stratCandleProps': ['open'], 'trades': False}}}
Error: config failure
http://localhost:3000/api/backtest
{'watch': {'exchange': 'bitfinex', 'currency': 'USD', 'asset': 'BTC'}, 'paperTrader': {'fee': 0.25, 'feeMaker': 0.15, 'feeTaker': 0.25, 'feeUsing': 'maker', 'slippage': 0.05, 'simulationBalance': {'asset': 0, 'currency': 100}, 'reportRoundtrips': True, 'enabled': True}, 'tradingAdvisor': {'enabled': True, 'method': 'Nico_RBB_ADX2_BB', 'candleSize': 5, 'historySize': 2000}, 'Nico_RBB_ADX2_BB': {'ADX': {'adx': 3.2, 'high': 41.0, 'low': 40.0}, 'BBands': {'NbDevDn': 2.0, 'NbDevUp': 2.0, 'TimePeriod': 20.0}, 'BBtrend': {'bearPersistence': 8.79, 'bullPersistence': 1.1400000000000001, 'lowerThreshold': 24.0, 'upperThreshold': 75.1}, 'BEAR': {'high': 52.0, 'low': 38.5, 'mod_high': -16.0, 'mod_low': -7.2, 'rsi': 7.8}, 'BULL': {'high': 89.0, 'low': 32.6, 'mod_high': 13.5, 'mod_low': -16.0, 'rsi': 7.8}, 'SMA': {'long': 650.0, 'short': 64.0}}, 'backtest': {'daterange': {'from': '2018-01-31 18:57:09', 'to': '2018-07-30 18:57:09'}}, 'performanceAnalyzer': {'riskFreeReturn': 2, 'enabled': True}, 'valid': True, 'data': {'candleProps': ['id', 'start', 'open', 'high', 'low', 'close', 'vwp', 'volume', 'trades'], 'indicatorResults': True, 'report': True, 'roundtrips': True, 'trades': True}, 'backtestResultExporter': {'enabled': True, 'writeToDisk': False, 'data': {'stratUpdates': False, 'roundtrips': True, 'stratCandles': False, 'stratCandleProps': ['open'], 'trades': False}}}
Error: config failure
http://localhost:3000/api/backtest
{'watch': {'exchange': 'bitfinex', 'currency': 'USD', 'asset': 'BTC'}, 'paperTrader': {'fee': 0.25, 'feeMaker': 0.15, 'feeTaker': 0.25, 'feeUsing': 'maker', 'slippage': 0.05, 'simulationBalance': {'asset': 0, 'currency': 100}, 'reportRoundtrips': True, 'enabled': True}, 'tradingAdvisor': {'enabled': True, 'method': 'Nico_RBB_ADX2_BB', 'candleSize': 5, 'historySize': 2000}, 'Nico_RBB_ADX2_BB': {'ADX': {'adx': 3.2, 'high': 42.0, 'low': 40.0}, 'BBands': {'NbDevDn': 2.0, 'NbDevUp': 2.0, 'TimePeriod': 20.0}, 'BBtrend': {'bearPersistence': 14.870000000000001, 'bullPersistence': 3.8000000000000003, 'lowerThreshold': 50.8, 'upperThreshold': 73.0}, 'BEAR': {'high': 54.6, 'low': 30.4, 'mod_high': -16.0, 'mod_low': -8.7, 'rsi': 7.8}, 'BULL': {'high': 89.6, 'low': 32.0, 'mod_high': 10.65, 'mod_low': -16.0, 'rsi': 10.04}, 'SMA': {'long': 860.0, 'short': 48.0}}, 'backtest': {'daterange': {'from': '2018-01-31 18:57:09', 'to': '2018-07-30 18:57:09'}}, 'performanceAnalyzer': {'riskFreeReturn': 2, 'enabled': True}, 'valid': True, 'data': {'candleProps': ['id', 'start', 'open', 'high', 'low', 'close', 'vwp', 'volume', 'trades'], 'indicatorResults': True, 'report': True, 'roundtrips': True, 'trades': True}, 'backtestResultExporter': {'enabled': True, 'writeToDisk': False, 'data': {'stratUpdates': False, 'roundtrips': True, 'stratCandles': False, 'stratCandleProps': ['open'], 'trades': False}}}
Error: config failure
http://localhost:3000/api/backtest
{'watch': {'exchange': 'bitfinex', 'currency': 'USD', 'asset': 'BTC'}, 'paperTrader': {'fee': 0.25, 'feeMaker': 0.15, 'feeTaker': 0.25, 'feeUsing': 'maker', 'slippage': 0.05, 'simulationBalance': {'asset': 0, 'currency': 100}, 'reportRoundtrips': True, 'enabled': True}, 'tradingAdvisor': {'enabled': True, 'method': 'Nico_RBB_ADX2_BB', 'candleSize': 5, 'historySize': 2000}, 'Nico_RBB_ADX2_BB': {'ADX': {'adx': 3.2, 'high': 35.0, 'low': 40.0}, 'BBands': {'NbDevDn': 2.0, 'NbDevUp': 2.0, 'TimePeriod': 20.0}, 'BBtrend': {'bearPersistence': 15.44, 'bullPersistence': 2.4000000000000004, 'lowerThreshold': 56.4, 'upperThreshold': 73.0}, 'BEAR': {'high': 52.0, 'low': 39.4, 'mod_high': -16.0, 'mod_low': -13.5, 'rsi': 14.84}, 'BULL': {'high': 87.2, 'low': 36.2, 'mod_high': 4.2, 'mod_low': -16.0, 'rsi': 7.8}, 'SMA': {'long': 650.0, 'short': 28.0}}, 'backtest': {'daterange': {'from': '2018-01-31 18:57:09', 'to': '2018-07-30 18:57:09'}}, 'performanceAnalyzer': {'riskFreeReturn': 2, 'enabled': True}, 'valid': True, 'data': {'candleProps': ['id', 'start', 'open', 'high', 'low', 'close', 'vwp', 'volume', 'trades'], 'indicatorResults': True, 'report': True, 'roundtrips': True, 'trades': True}, 'backtestResultExporter': {'enabled': True, 'writeToDisk': False, 'data': {'stratUpdates': False, 'roundtrips': True, 'stratCandles': False, 'stratCandleProps': ['open'], 'trades': False}}}
Error: config failure
http://localhost:3000/api/backtest
{'watch': {'exchange': 'bitfinex', 'currency': 'USD', 'asset': 'BTC'}, 'paperTrader': {'fee': 0.25, 'feeMaker': 0.15, 'feeTaker': 0.25, 'feeUsing': 'maker', 'slippage': 0.05, 'simulationBalance': {'asset': 0, 'currency': 100}, 'reportRoundtrips': True, 'enabled': True}, 'tradingAdvisor': {'enabled': True, 'method': 'Nico_RBB_ADX2_BB', 'candleSize': 5, 'historySize': 2000}, 'Nico_RBB_ADX2_BB': {'ADX': {'adx': 3.2, 'high': 45.5, 'low': 40.0}, 'BBands': {'NbDevDn': 2.0, 'NbDevUp': 2.0, 'TimePeriod': 20.0}, 'BBtrend': {'bearPersistence': 15.44, 'bullPersistence': 2.4000000000000004, 'lowerThreshold': 56.4, 'upperThreshold': 85.3}, 'BEAR': {'high': 55.4, 'low': 39.4, 'mod_high': -16.0, 'mod_low': -13.5, 'rsi': 14.84}, 'BULL': {'high': 87.2, 'low': 36.2, 'mod_high': 4.2, 'mod_low': -16.0, 'rsi': 7.8}, 'SMA': {'long': 650.0, 'short': 28.0}}, 'backtest': {'daterange': {'from': '2018-01-31 18:57:09', 'to': '2018-07-30 18:57:09'}}, 'performanceAnalyzer': {'riskFreeReturn': 2, 'enabled': True}, 'valid': True, 'data': {'candleProps': ['id', 'start', 'open', 'high', 'low', 'close', 'vwp', 'volume', 'trades'], 'indicatorResults': True, 'report': True, 'roundtrips': True, 'trades': True}, 'backtestResultExporter': {'enabled': True, 'writeToDisk': False, 'data': {'stratUpdates': False, 'roundtrips': True, 'stratCandles': False, 'stratCandleProps': ['open'], 'trades': False}}}
Error: config failure
http://localhost:3000/api/backtest
{'watch': {'exchange': 'bitfinex', 'currency': 'USD', 'asset': 'BTC'}, 'paperTrader': {'fee': 0.25, 'feeMaker': 0.15, 'feeTaker': 0.25, 'feeUsing': 'maker', 'slippage': 0.05, 'simulationBalance': {'asset': 0, 'currency': 100}, 'reportRoundtrips': True, 'enabled': True}, 'tradingAdvisor': {'enabled': True, 'method': 'Nico_RBB_ADX2_BB', 'candleSize': 5, 'historySize': 2000}, 'Nico_RBB_ADX2_BB': {'ADX': {'adx': 3.2, 'high': 35.0, 'low': 40.0}, 'BBands': {'NbDevDn': 2.0, 'NbDevUp': 2.0, 'TimePeriod': 20.0}, 'BBtrend': {'bearPersistence': 8.79, 'bullPersistence': 1.1400000000000001, 'lowerThreshold': 24.0, 'upperThreshold': 75.1}, 'BEAR': {'high': 52.0, 'low': 13.0, 'mod_high': -16.0, 'mod_low': -7.2, 'rsi': 7.8}, 'BULL': {'high': 89.0, 'low': 32.6, 'mod_high': 13.5, 'mod_low': -16.0, 'rsi': 7.8}, 'SMA': {'long': 650.0, 'short': 64.0}}, 'backtest': {'daterange': {'from': '2018-01-31 18:57:09', 'to': '2018-07-30 18:57:09'}}, 'performanceAnalyzer': {'riskFreeReturn': 2, 'enabled': True}, 'valid': True, 'data': {'candleProps': ['id', 'start', 'open', 'high', 'low', 'close', 'vwp', 'volume', 'trades'], 'indicatorResults': True, 'report': True, 'roundtrips': True, 'trades': True}, 'backtestResultExporter': {'enabled': True, 'writeToDisk': False, 'data': {'stratUpdates': False, 'roundtrips': True, 'stratCandles': False, 'stratCandleProps': ['open'], 'trades': False}}}
Error: config failure
http://localhost:3000/api/backtest
{'watch': {'exchange': 'bitfinex', 'currency': 'USD', 'asset': 'BTC'}, 'paperTrader': {'fee': 0.25, 'feeMaker': 0.15, 'feeTaker': 0.25, 'feeUsing': 'maker', 'slippage': 0.05, 'simulationBalance': {'asset': 0, 'currency': 100}, 'reportRoundtrips': True, 'enabled': True}, 'tradingAdvisor': {'enabled': True, 'method': 'Nico_RBB_ADX2_BB', 'candleSize': 5, 'historySize': 2000}, 'Nico_RBB_ADX2_BB': {'ADX': {'adx': 3.18, 'high': 35.0, 'low': 40.0}, 'BBands': {'NbDevDn': 2.0, 'NbDevUp': 2.0, 'TimePeriod': 20.0}, 'BBtrend': {'bearPersistence': 8.79, 'bullPersistence': 1.1400000000000001, 'lowerThreshold': 24.8, 'upperThreshold': 75.1}, 'BEAR': {'high': 52.0, 'low': 38.5, 'mod_high': -2.3999999999999986, 'mod_low': -7.2, 'rsi': 9.64}, 'BULL': {'high': 72.0, 'low': 32.0, 'mod_high': 13.5, 'mod_low': 8.8, 'rsi': 14.2}, 'SMA': {'long': 1295.0, 'short': 64.0}}, 'backtest': {'daterange': {'from': '2018-01-31 18:57:09', 'to': '2018-07-30 18:57:09'}}, 'performanceAnalyzer': {'riskFreeReturn': 2, 'enabled': True}, 'valid': True, 'data': {'candleProps': ['id', 'start', 'open', 'high', 'low', 'close', 'vwp', 'volume', 'trades'], 'indicatorResults': True, 'report': True, 'roundtrips': True, 'trades': True}, 'backtestResultExporter': {'enabled': True, 'writeToDisk': False, 'data': {'stratUpdates': False, 'roundtrips': True, 'stratCandles': False, 'stratCandleProps': ['open'], 'trades': False}}}
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "C:\Users\Nicolas\AppData\Local\Programs\Python\Python36\lib\multiprocessing\pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "C:\Users\Nicolas\AppData\Local\Programs\Python\Python36\lib\multiprocessing\pool.py", line 47, in starmapstar
    return list(itertools.starmap(args[0], args[1]))
  File "C:\Gekko\japonicus\evolution_generations.py", line 52, in standardEvaluate
    genconf, Datasets, phenotype, gekkoUrl
  File "C:\Gekko\japonicus\evaluation\gekko\backtest.py", line 76, in Evaluate
    for Dataset in Datasets
  File "C:\Gekko\japonicus\evaluation\gekko\backtest.py", line 76, in <listcomp>
    for Dataset in Datasets
  File "C:\Gekko\japonicus\evaluation\gekko\backtest.py", line 46, in runBacktest
    if type(result['performanceReport']) is bool:
KeyError: 'performanceReport'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "../../Gekko/japonicus/japonicus.py", line 175, in <module>
    launchJaponicus(settings, options)
  File "../../Gekko/japonicus/japonicus.py", line 159, in launchJaponicus
    EvaluationMode, settings, options, web=web_server
  File "C:\Gekko\japonicus\evolution_generations.py", line 247, in gekko_generations
    World.runEPOCH()
  File "C:\Gekko\japonicus\promoterz\sequence\parallel_world.py", line 13, in world_EPOCH
    LOCALE.run()
  File "C:\Gekko\japonicus\promoterz\locale.py", line 40, in run
    self.loop(self.World, self)
  File "C:\Gekko\japonicus\promoterz\sequence\standard_loop.py", line 41, in standard_loop
    locale
  File "C:\Gekko\japonicus\promoterz\evaluationPool.py", line 78, in evaluatePopulation
    results[A] = results[A].get(timeout=timeout)
  File "C:\Users\Nicolas\AppData\Local\Programs\Python\Python36\lib\multiprocessing\pool.py", line 644, in get
    raise self._value
  File "C:\Users\Nicolas\AppData\Local\Programs\Python\Python36\lib\multiprocessing\pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "C:\Gekko\japonicus\promoterz\evaluationPool.py", line 47, in evaluateBackend
    fitnesses = P.starmap(self.EvaluationTool, Q)
  File "C:\Users\Nicolas\AppData\Local\Programs\Python\Python36\lib\multiprocessing\pool.py", line 274, in starmap
    return self._map_async(func, iterable, starmapstar, chunksize).get()
  File "C:\Users\Nicolas\AppData\Local\Programs\Python\Python36\lib\multiprocessing\pool.py", line 644, in get
    raise self._value
KeyError: 'performanceReport'

Any help would be greatly appreciated :)

Gab0 commented 6 years ago

@nmikaty pushed a fix that ignores those errors, maybe it solves everything. Those parameters shown on the error for your strategy are actually legal? cheers!

MeafQ commented 6 years ago

@Gab0 I have the same problem, but mine could crash on the first epoch. I'm using neuralnet by zschro (with little edits to eliminate text parametrs). After the update it produces 0 or 1 trades. And a lot of prints 'performanceReport'. Before that same parameters wouldn't even run if I don't find right numbers for them.

Mainly I have a problem with threshold_buy and threshold_sell and momentum if I use a method that require it. I would need to change them to like [1,1] so japonicus could be run normally.

I run it only with -g --strat. Here is a log (I removed all 'performanceReport', but trust me there was a lot):

The profits reported here depends on backtest interpreter function;
        interpreter v3:
if <backtest profit> > 0: <shown profit> = <backtest profit> - <market profit>
else <shown profit> = <backtest profit>

found gekko @ http://localhost:3000
C:/Gekko/japonicus/japonicus.py -g --strat neuralnet
Evolving neuralnet strategy;

evaluated parameters ranges:
threshold_buy                 [0.1, 3.0]

threshold_sell                [-3.0, -0.1]

method                        [0, 0]

learning_rate                 [1.0, 1.0]

momentum                      [1.0, 1.0]

decay                         [0.01, 1.0]

stoploss_enabled              [1, 1]

stoploss_threshold            [0.7, 1.0]

hodle_threshold               [0.7, 1.0]

price_buffer_len              [10, 300]

min_predictions               [20, 3000]

evolution candlestick dataset 2018-07-01 17:49:00 to 2018-08-03 17:48:00
BNB/USDT @binance

evolution candlestick dataset 2018-07-01 17:49:00 to 2018-08-03 17:48:00
BNB/USDT @binance

evaluation candlestick dataset 2018-06-30 23:21:00 to 2018-08-02 23:19:00
LTC/USDT @binance

        ======  EPOCH 0/3000  ======
Locale1
first unevaluated: 50
0 individues removed due to equality
[50]
Population dead after trading number filter.
Population dead after roundtrip exposure filter.
Repopulating... Aborting epoch.
EPOCH 0 &50
Maximum profit 0         Average profit 0
Minimum profit 0         Profit variation 0
Population size 50       Max population size 50
Avg trade number 0       Avg sharpe ratio 0
Avg exposure time 0

Locale2
first unevaluated: 50
0 individues removed due to equality
[50]
Population dead after trading number filter.
Population dead after roundtrip exposure filter.
Repopulating... Aborting epoch.
EPOCH 0 &50
Maximum profit 0         Average profit 0
Minimum profit 0         Profit variation 0
Population size 50       Max population size 50
Avg trade number 0       Avg sharpe ratio 0
Avg exposure time 0

Locale3
first unevaluated: 50
0 individues removed due to equality
[50]
Population dead after trading number filter.
Population dead after roundtrip exposure filter.
Repopulating... Aborting epoch.
EPOCH 0 &50
Maximum profit 3.946     Average profit 3.946
Minimum profit 3.946     Profit variation 0.000
Population size 50       Max population size 50
Avg trade number 1       Avg sharpe ratio 0
Avg exposure time 0

Epoch runs in 422.95 seconds;

        ======  EPOCH 1/3000  ======
Locale1
first unevaluated: 70
9 individues removed due to equality
[61]
C:\Users\Vadim\AppData\Local\Programs\Python\Python36-32\lib\site-packages\plotly\tools.py:102: UserWarning:

Looks like you don't have 'read-write' permission to your 'home' ('~') directory or to our '~/.plotly' directory. That means plotly's python api can't setup local configuration files. No problem though! You'll just have to sign-in using 'plotly.plotly.sign_in()'. For help with that: 'help(plotly.plotly.sign_in)'.
Questions? Visit https://support.plot.ly

C:\Users\Vadim\AppData\Local\Programs\Python\Python36-32\lib\site-packages\plotly\tools.py:102: UserWarning:

Looks like you don't have 'read-write' permission to your 'home' ('~') directory or to our '~/.plotly' directory. That means plotly's python api can't setup local configuration files. No problem though! You'll just have to sign-in using 'plotly.plotly.sign_in()'. For help with that: 'help(plotly.plotly.sign_in)'.
Questions? Visit https://support.plot.ly

Population dead after trading number filter.
Population dead after roundtrip exposure filter.
Repopulating... Aborting epoch.
EPOCH 1 &61
Maximum profit 0         Average profit 0
Minimum profit 0         Profit variation 0
Population size 61       Max population size 50
Avg trade number 0       Avg sharpe ratio 0
Avg exposure time 0

C:\Gekko\japonicus\promoterz\supplement\PRoFIGA.py:6: RuntimeWarning:

invalid value encountered in double_scalars

POP_SIZE PROFIGA ERROR;
Locale2
first unevaluated: 60
12 individues removed due to equality
[48]
C:\Users\Vadim\AppData\Local\Programs\Python\Python36-32\lib\site-packages\plotly\tools.py:102: UserWarning:

Looks like you don't have 'read-write' permission to your 'home' ('~') directory or to our '~/.plotly' directory. That means plotly's python api can't setup local configuration files. No problem though! You'll just have to sign-in using 'plotly.plotly.sign_in()'. For help with that: 'help(plotly.plotly.sign_in)'.
Questions? Visit https://support.plot.ly

Population dead after trading number filter.
Population dead after roundtrip exposure filter.
Repopulating... Aborting epoch.
EPOCH 1 &48
Maximum profit 0         Average profit 0
Minimum profit 0         Profit variation 0
Population size 48       Max population size 50
Avg trade number 0       Avg sharpe ratio 0
Avg exposure time 0

POP_SIZE PROFIGA ERROR;
Locale3
first unevaluated: 63
12 individues removed due to equality
[51]
Population dead after trading number filter.
Population dead after roundtrip exposure filter.
Repopulating... Aborting epoch.
EPOCH 1 &51
Maximum profit 3.946     Average profit 3.946
Minimum profit 3.946     Profit variation 0.000
Population size 51       Max population size 50
Avg trade number 1       Avg sharpe ratio 0
Avg exposure time 0

Epoch runs in 478.38 seconds;

        ======  EPOCH 2/3000  ======
Locale1
first unevaluated: 55
14 individues removed due to equality
[41]
C:\Users\Vadim\AppData\Local\Programs\Python\Python36-32\lib\site-packages\plotly\tools.py:102: UserWarning:

Looks like you don't have 'read-write' permission to your 'home' ('~') directory or to our '~/.plotly' directory. That means plotly's python api can't setup local configuration files. No problem though! You'll just have to sign-in using 'plotly.plotly.sign_in()'. For help with that: 'help(plotly.plotly.sign_in)'.
Questions? Visit https://support.plot.ly

Population dead after trading number filter.
Population dead after roundtrip exposure filter.
Repopulating... Aborting epoch.
EPOCH 2 &41
Maximum profit 0         Average profit 0
Minimum profit 0         Profit variation 0
Population size 41       Max population size 30
Avg trade number 0       Avg sharpe ratio 0
Avg exposure time 0

POP_SIZE PROFIGA ERROR;
Locale2
first unevaluated: 42
13 individues removed due to equality
[29]
Population dead after trading number filter.
Population dead after roundtrip exposure filter.
Repopulating... Aborting epoch.
EPOCH 2 &29
Maximum profit 0         Average profit 0
Minimum profit 0         Profit variation 0
Population size 29       Max population size 30
Avg trade number 0       Avg sharpe ratio 0
Avg exposure time 0

POP_SIZE PROFIGA ERROR;
Locale3
first unevaluated: 62
9 individues removed due to equality
[53]
Population dead after trading number filter.
Population dead after roundtrip exposure filter.
Repopulating... Aborting epoch.
EPOCH 2 &53
Maximum profit 3.946     Average profit 3.946
Minimum profit 3.946     Profit variation 0.000
Population size 53       Max population size 50
Avg trade number 1       Avg sharpe ratio 0
Avg exposure time 0

Epoch runs in 328.73 seconds;

        ======  EPOCH 3/3000  ======
Locale1
first unevaluated: 48
8 individues removed due to equality
[40]
MeafQ commented 6 years ago

neuralnet.zip My version of neuralnet if needed.

Gab0 commented 6 years ago

@MeafQ Hey can you run with a non-nn strategy? If you are on gekko v0.6 (required), jap could be passing parameters that crashes the strategy. We don't have a single working backtest here, right?

MeafQ commented 6 years ago

@Gab0 I'm on the last stable version of Gekko. Some simplier strategy will work better. I tried PPO it was fine but at some point I saw that japonicus dissapeared. I think it also crashed but more rarely, I assume it's all about the chance. While japonicus refuses to work with some parameters I find that backtest straight in Gekko is just fine.

Also to be clear I use more defaultish parameters for backtest: 3 locales, 30 days, 50 population size.

nmikaty commented 6 years ago

Hi everyone,

@Gab0 yes the parameters I use are legal for this strategy. And before crashing it had run thousands of successful backtests, so I don't think there is anything wrong with the strategy parameters.

I pulled your last edit and tried a new run. This time I got stopped at epoch 30, after what I believe was the end of the first evaluation break (evaluateSettingsPeriodically = 30) with this new error:

 Testing Strategy of Locale11 @ EPOCH 27:

Test on random candles...  relativeProfit: -10.362      nbTrades: 289.0 sharpe: 164.56
                2017-05-13 19:28:40 to 2017-11-09 19:28:40      BTC/USD @bitfinex

Test on random candles...  relativeProfit: -38.857      nbTrades: 299.0 sharpe: 131.70
                2017-04-03 00:42:58 to 2017-09-30 00:42:58      BTC/USD @bitfinex

Test on random candles...  relativeProfit: 178.374      nbTrades: 260.0 sharpe: 234.58
                2017-07-24 07:13:35 to 2018-01-20 07:13:35      BTC/USD @bitfinex

Test on random candles...  relativeProfit: -4.281       nbTrades: 297.0 sharpe: -4.18
                2018-01-13 21:12:17 to 2018-07-12 21:12:17      BTC/USD @bitfinex

Test on random candles...  relativeProfit: 412.790      nbTrades: 275.0 sharpe: 234.04
                2017-09-22 14:05:28 to 2018-03-21 14:05:28      BTC/USD @bitfinex

Test on random candles...  relativeProfit: 200.803      nbTrades: 285.0 sharpe: 102.18
                2017-11-12 19:21:04 to 2018-05-11 19:21:04      BTC/USD @bitfinex

Test on random candles...  relativeProfit: 104.324      nbTrades: 279.0 sharpe: 178.33
                2017-05-25 10:44:03 to 2017-11-21 10:44:03      BTC/USD @bitfinex

Test on random candles...  relativeProfit: 156.229      nbTrades: 291.0 sharpe: 306.69
                2017-06-23 14:55:09 to 2017-12-20 14:55:09      BTC/USD @bitfinex

Test on random candles...  relativeProfit: -55.604      nbTrades: 296.0 sharpe: 141.66
                2017-02-15 12:11:03 to 2017-08-14 12:11:03      BTC/USD @bitfinex

Test on random candles...  relativeProfit: 330.792      nbTrades: 282.0 sharpe: 195.03
                2017-09-03 09:30:48 to 2018-03-02 09:30:48      BTC/USD @bitfinex

Test on random candles...  relativeProfit: -92.029      nbTrades: 284.0 sharpe: 116.97
                2017-04-15 01:31:35 to 2017-10-12 01:31:35      BTC/USD @bitfinex

Test on random candles...  relativeProfit: -14.800      nbTrades: 300.0 sharpe: 122.11
                2017-05-17 21:41:44 to 2017-11-13 21:41:44      BTC/USD @bitfinex

Relative profit on evolution dataset: 1167.379
Current parameters are viable.

Relative profit on evaluation dataset:
        Test on random candles...  relativeProfit: 1890.051     nbTrades: 885.0 sharpe: 474.68
~~~~~~~~~~~~~~~~~~
 1167.379 final profit ~~~~
 -- Settings for Gekko config.js --
{
  "ADX": {
    "adx": 3.2,
    "high": 44.5,
    "low": 40.0
  },
  "BBands": {
    "NbDevDn": 2.0,
    "NbDevUp": 2.0,
    "TimePeriod": 20.0
  },
  "BBtrend": {
    "bearPersistence": 11.45,
    "bullPersistence": 2.4000000000000004,
    "lowerThreshold": 39.6,
    "upperThreshold": 89.2
  },
  "BEAR": {
    "high": 52.0,
    "low": 25.9,
    "mod_high": -3.1999999999999993,
    "mod_low": -6.15,
    "rsi": 7.8
  },
  "BULL": {
    "high": 83.2,
    "low": 41.0,
    "mod_high": 3.9,
    "mod_low": -18.0,
    "rsi": 12.84
  },
  "SMA": {
    "long": 650.0,
    "short": 28.0
  }
}
 -- Settings for Gekko --ui webpage --
[ADX]
adx = 3.2
high = 44.5
low = 40.0

[BBands]
NbDevDn = 2.0
NbDevUp = 2.0
TimePeriod = 20.0

[BBtrend]
bearPersistence = 11.45
bullPersistence = 2.4000000000000004
lowerThreshold = 39.6
upperThreshold = 89.2

[BEAR]
high = 52.0
low = 25.9
mod_high = -3.1999999999999993
mod_low = -6.15
rsi = 7.8

[BULL]
high = 83.2
low = 41.0
mod_high = 3.9
mod_low = -18.0
rsi = 12.84

[SMA]
long = 650.0
short = 28.0

Remember to check MAX and MIN values for each parameter.
        results may improve with extended ranges.
Traceback (most recent call last):
  File "../../Gekko/japonicus/japonicus.py", line 198, in <module>
    launchJaponicus(settings, options)
  File "../../Gekko/japonicus/japonicus.py", line 181, in launchJaponicus
    EvaluationMode, settings, options, web=web_server
  File "C:\Gekko\japonicus\evolution_generations.py", line 250, in gekko_generations
    evaluationBreak.showResults(World)
  File "C:\Gekko\japonicus\evaluationBreak.py", line 123, in showResults
    if World.EvolutionStatistics:
AttributeError: 'World' object has no attribute 'EvolutionStatistics'

Thank you for your help!

Gab0 commented 6 years ago

@nmikaty Oh, this seems to be a bug on evaluationBreak, I'm uploading a fix in a few mins.

@MeafQ This strategy is simply not trading much, you should disable min trade count filter at settings/generations.toml to GA this strategy.

MeafQ commented 6 years ago

@Gab0 Thank you for the tip. Also a small bug with plotly, it repeatedly asks about credentials as user warning even without enabled web interface. This is not very crucial but those warnings are very frequent. Maybe plotly and dash could be used in full offline mode? Of course with all needed scripts beeing stored localy.

Offtopic. I found out that AI can optimize strategy in a very unusable way. Yes, it does find the most profitable settings for a particular dataset, but only for that dataset, in this range, with those candles etc. I adjusted my settings ranges, but it still not enough. I think it needs to be checked in more variation of situation, so for AI it would be impossible to find those unlogical 'gaps' in settings just because it's backtesting and it can run the same dataset endless times. So I'm trying to add more Locals and manually set every of them to a different dataset (including eval_dataset). Will it help? Or there is something that could be done else.

Also is there any place to discuss about japonicus, like discord server? Except official gekko one, because there would be hard to find someone who knows something.

nmikaty commented 6 years ago

Thank you @Gab0, I am running a new test now and will edit this post tomorrow with the result.

@MeafQ I have the same warning about plotly. This seems to be related to user permissions on Windows folders. I tried to find a solution a while back but never succeeded. Here we were talking about it briefly: https://github.com/Gab0/japonicus/issues/97#event-1561239269 Please update if you find a solution!

About your other subject, yes optimizing parameters for a particular dataset will give you over-fitting parameters. Here is what I'm doing:

  1. One set of parameters for each pair (BTCUSD for example) and candle size. Because I believe each pair has its own behavior (probably mostly due to volume so it might be overcome?).

  2. Select large datasets to optimize parameters. 1 to 3 years. If your parameters are over-fitted for 2 years of data, you are less prone to surprises for the next couple of months. I usually go with at least 1.5 year of data for a strategy on 5 minutes candles.

  3. I try to cover the totality of my dataset with locales. For instance right now I am running Japonicus on a 600 days dataset (1.7 year), with evaluations of 90 days. So I use 20 locales, which covers 20*90 = 1800 days, 3 times my dataset. If I understand correctly (@Gab0 correct me if I'm wrong) the locales are spawned at random times in the dataset. So this hopefully covers most of the dataset. Of course it can take time...

  4. Re-optimize parameters every couple of months to adapt to changes in behavior over time.

I have been playing with Japonicus for less than a year, and running live Gekkos only for a few months with up and downs. So I'm not sure I'm on the right path. Just giving my thoughts on the subject.

I also agree with you that it would be nice to have a place to talk about Japonicus. Also, I have so many questions about how everything works (locales, reproduction, migration of individuals, etc), it would be fantastic to be able to do a live question/answer kind of stuff. On discord or other... hey @Gab0 ? :)

Gab0 commented 6 years ago

@nmikaty Yeah, locales spawn at random times with a random piece of the dataset...

The thing about overfitting is that it depends a lot on the strat. They say the simple ones are the best but I disagree. A 'dumb' strategy like gekko's sample MACD thing just goes: if MACD.result > threshold.up: buy, or something like that. This thing will surely overfit quickly, as given set of parameters, which are five for this strategy, simply cannot put up to many market variations and can do good on a small timeframe but will lose everything in different candles/market situations. If you run a GA with these simpler strats on a big timeframe the results will show the inability to profit on long term with same parameters.

A more complex strat like RSI_BULL_BEAR, in other hand, can adapt easier with same parameters to different types of markets by having an RSI ensenbly with their own parameters to BULL and BEAR situations, delaying the overfit.

The @nmikaty method of GA is good, but of course needs a good strategy behind to get any goodz.

Well, I thing the community on jap is not big enough to hold a discord channel, if I'm wrong we can have a channel ofc and I will anwser any kind of question about the project, here or there XD cheers!

nmikaty commented 6 years ago

@Gab0 I ran a new test and got stopped again at what looks like the end (but could be somewhere in the middle) of the first evaluation break (epoch 30) with a new error:

Relative profit on evaluation dataset:
        Test on random candles...  relativeProfit: -26.865      nbTrades: 429.0 sharpe: 97.82
~~~~~~~~~~~~~~~~~~
 -240.156 final profit ~~~~
 -- Settings for Gekko config.js --
{
  "ADX": {
    "active": 0.0,
    "adx": 4.04,
    "high": 64.5,
    "low": 40.0
  },
  "BBands": {
    "NbDevDn": 2.0,
    "NbDevUp": 2.0,
    "TimePeriod": 20.0
  },
  "BBtrend": {
    "active": 0.0,
    "bearPersistence": 18.29,
    "bullPersistence": 2.4000000000000004,
    "lowerThreshold": 24.0,
    "upperThreshold": 82.6
  },
  "BEAR": {
    "high": 44.0,
    "low": 9.2,
    "mod_high": 6.4,
    "mod_low": -13.5,
    "rsi": 7.8
  },
  "BULL": {
    "high": 94.3,
    "low": 32.8,
    "mod_high": 1.5,
    "mod_low": -18.0,
    "rsi": 12.280000000000001
  },
  "SMA": {
    "long": 1340.0,
    "short": 28.0
  }
}
 -- Settings for Gekko --ui webpage --
[ADX]
active = 0.0
adx = 4.04
high = 64.5
low = 40.0

[BBands]
NbDevDn = 2.0
NbDevUp = 2.0
TimePeriod = 20.0

[BBtrend]
active = 0.0
bearPersistence = 18.29
bullPersistence = 2.4000000000000004
lowerThreshold = 24.0
upperThreshold = 82.6

[BEAR]
high = 44.0
low = 9.2
mod_high = 6.4
mod_low = -13.5
rsi = 7.8

[BULL]
high = 94.3
low = 32.8
mod_high = 1.5
mod_low = -18.0
rsi = 12.280000000000001

[SMA]
long = 1340.0
short = 28.0

Remember to check MAX and MIN values for each parameter.
        results may improve with extended ranges.
0 selected;
0 selected;
Selecting 1+0 individues, random test;
Traceback (most recent call last):
  File "../../Gekko/japonicus/japonicus.py", line 198, in <module>
    launchJaponicus(settings, options)
  File "../../Gekko/japonicus/japonicus.py", line 181, in launchJaponicus
    EvaluationMode, settings, options, web=web_server
  File "C:\Gekko\japonicus\evolution_generations.py", line 250, in gekko_generations
    evaluationBreak.showResults(World)
  File "C:\Gekko\japonicus\evaluationBreak.py", line 135, in showResults
    show=False, replace=False)
  File "C:\Gekko\japonicus\promoterz\logger.py", line 30, in log
    self.__dict__[target] += message + '\n'
TypeError: unsupported operand type(s) for +: 'dict' and 'str'

Also, at random times I run into this error without stopping the execution:

Locale17
first unevaluated: 51
21 individues removed due to equality
[30]
Error: config failure
('Connection aborted.', OSError(10048, 'Une seule utilisation de chaque adresse de socket (protocole/adresse réseau/port) est habituellement autorisée', None, 10048, None))
http://localhost:3000/api/backtest
{'watch': {'exchange': 'bitfinex', 'currency': 'USD', 'asset': 'BTC'}, 'paperTrader': {'fee': 0.25, 'feeMaker': 0.15, 'feeTaker': 0.25, 'feeUsing': 'maker', 'slippage': 0.05, 'simulationBalance': {'asset': 0, 'currency': 100}, 'reportRoundtrips': True, 'enabled': True}, 'tradingAdvisor': {'enabled': True, 'method': 'Nico_RBB_ADX2_BB', 'candleSize': 5, 'historySize': 2000}, 'Nico_RBB_ADX2_BB': {'ADX': {'active': 0.0, 'adx': 3.56, 'high': 71.5, 'low': 40.0}, 'BBands': {'NbDevDn': 2.0, 'NbDevUp': 2.0, 'TimePeriod': 20.0}, 'BBtrend': {'active': 0.0, 'bearPersistence': 4.8, 'bullPersistence': 5.9, 'lowerThreshold': 31.6, 'upperThreshold': 97.9}, 'BEAR': {'high': 70.0, 'low': 16.2, 'mod_high': 15.0, 'mod_low': -4.950000000000001, 'rsi': 8.76}, 'BULL': {'high': 79.25, 'low': 53.6, 'mod_high': 7.6499999999999995, 'mod_low': -18.2, 'rsi': 11.16}, 'SMA': {'long': 1580.0, 'short': 70.4}}, 'backtest': {'daterange': {'from': '2018-03-06 17:01:43', 'to': '2018-06-04 17:01:43'}}, 'performanceAnalyzer': {'riskFreeReturn': 2, 'enabled': True}, 'valid': True, 'data': {'candleProps': ['id', 'start', 'open', 'high', 'low', 'close', 'vwp', 'volume', 'trades'], 'indicatorResults': True, 'report': True, 'roundtrips': True, 'trades': True}, 'backtestResultExporter': {'enabled': True, 'writeToDisk': False, 'data': {'stratUpdates': False, 'roundtrips': True, 'stratCandles': False, 'stratCandleProps': ['open'], 'trades': False}}}
'bool' object is not subscriptable
EPOCH 27        &30
Maximum profit 55.734      Average profit 27.235
Minimum profit -48.821     Profit variation 32.215
Population size 137        Max population size 150
Avg trade number 117.700   Avg sharpe ratio 14.401
Avg exposure time 76.535

Maybe I will ask my questions about Japonicus behavior here then :) Or we could try and ask people if they would be interested in a Discord channel. Maybe raising a new issue and wait for some feedback?

About strategy, I use a modified version of RSI_BULL_BEAR, with my own blend of ADX modifier, and a BBand module. It gives great results on backtests, and is now running live with mitigated results (profitable but with big scary drawbacks). I am also working on a machine learning strategy with Neataptic ( https://wagenaartje.github.io/neataptic/ ), with indicators as inputs. Work in progress, I will update if anything interesting.

Cheers and thank you @Gab0 for your time!

MeafQ commented 6 years ago

@nmikaty @Gab0 Thank you for the tips. I have got the same error. It happens after saving results.

About plotly. I tried to at least eliminate warnings when dash/flask are not used, but maybe for some reason it helped for both circumstances. Test it. > japonicus.py changes

MeafQ commented 6 years ago

As I understand now about over-fitting. It makes sense to make more locales of the same dataset (with random time spawn etc), right? But what about eval_dataset? What does it do? Also if we think that every pair is different (at some degree at least), then you can think not only about how to get the best settings but also about which pair is the best for your strategy. Is there any way than just guessing and be running manually each pair test to compare results? Same thing with candle sizes.

Gab0 commented 6 years ago

Oh, I'm fixing this last bug.

@MeafQ the eval_dataset is the one used at evalbreaks. Well, the statistics per pair would be surely interesting but somewhat complicated to implement, maybe at next version x)

@nmikaty Yeah man, I also went to live trading for a while. No matter how good the results appeared on the backteststs, got no solid profit. Even ran one gekko for each pair @ binance simultaneously. Maybe this one independent bot per pair logic we are used too flawed to profitz... a integrated approach like they do with stocks at Quantopian where a single strategy runs every pair and even distribute the treasure between the assets might do better. We would need to rewrite gekko though.

nmikaty commented 6 years ago

@Gab0 I finally got time to pull your last changes and run a new test. Looks like everything is good now! I think you can close this issue 👍

I still like the idea of one set of parameters for each pair, so one bot for each pair is fine for me. Of course it would be easier to run one instance of Gekko on multiple pairs with different parameters... But that's just luxury :) I think that one of the key elements of trading/investing is to limit the risks by investing on as many assets as possible. Of course playing on only one pair is risky. And even if we were to trade on 100 crypto assets, as they tend to move all together we wouldn't be much safer. But it still gives us a really nice playfield to test strategies :) I also think the next best thing for Gekko would be the possibility to buy/sell with only part of our capital, so that we can invest in several assets with the same currency, or adapt the amounts invested to the confidence we have in our signals.

@MeafQ I will try your trick for the Plotly warnings on my next run. Thanks.

nmikaty commented 6 years ago

Oops, just one last bug @Gab0 . Nothing too bad. After a run, at the beginning of the log file japonicus.log the summary of the evaluation results is missing:

[...]
evolution candlestick dataset 2016-12-09 04:31:00 to 2018-08-01 04:31:00
BTC/USD @bitfinex

evolution candlestick dataset 2016-12-09 04:31:00 to 2018-08-01 04:31:00
BTC/USD @bitfinex

evaluation candlestick dataset 2016-12-09 04:31:00 to 2018-08-01 04:31:00
BTC/USD @bitfinex

evaluation  secondary   filenamev   e   i
v   e   i
v   e   i
v   e   i
v   e   i
v   e   i
v   e   i
v   e   i
v   e   i
v   e   i
v   e   i
v   e   i
v   e   i
v   e   i
v   e   i
v   e   i
v   e   i
v   e   i
v   e   i
v   e   i
v   e   i
v   e   i
v   e   i
v   e   i
v   e   i
v   e   i
[...]

We can still see the results by browsing through the entire file though.

Cheers and thank you again.