search

Gab0 / japonicus

Genetic Algorithm for Gekko Trading Bot.
MIT License
283 stars 103 forks source link

Connection problem with gekko ? #58

Closed grogcw closed 6 years ago

grogcw commented 6 years ago

Hi !

I'm new to this amazing projet, and I'm trying quite hard to make it work and understand its ways, but I'm facing a problem right now : it fails after 1m30 or so.

The main problem seems to be the connection between japonicus and gekko.

I've checked my config files (probably not THAT ONE which is causing this issue) to modify the url : http://localhost:3000/api/backtest to http://ip :3000/api/backtest

The weird thing still is that I think japonicus can actually GET data from gekko as seen in logs below, but can't SEND a test, am I right on this one ?

Thanks for helping,

Cheers !

JAPONICUS OUTPUT :

Locale2
first unevaluated: 5
0 individues removed due to equality
[5]
EPOCH 1 &5
Maximum profit 0.294    Average profit -7.287
Minimum profit -12.777  Profit variation 3.748
Population size 37  Max population size 30
Avg trade number 0.300  Avg sharpe ratio 0

Elder dies 0

Epoch runs in 108.09 seconds;

Selecting 1+4 individues, random test;
4 selected;
1 selected;
Error: config failure
http://localhost:3000/api/backtest
{'data': {'trades': True, 'candleProps': ['id', 'start', 'open', 'high', 'low', 'close', 'vwp', 'volume', 'trades'], 'report': True, 'roundtrips': False, 'indicatorResults': True}, 'gekkoConfig': {'valid': True, 'tradingAdvisor': {'candleSize': 15, 'method': 'japonicus/ja354DEMARSI', 'enabled': True, 'historySize': 10}, 'performanceAnalyzer': {'enabled': True, 'riskFreeReturn': 2}, 'debug': True, 'info': True, 'watch': {'exchange': 'binance', 'currency': 'BTC', 'asset': 'CND'}, 'paperTrader': {'reportRoundtrips': True, 'feeUsing': 'maker', 'simulationBalance': {'currency': 100, 'asset': 1}, 'slippage': 0.05, 'fee': 0.25, 'enabled': True, 'feeTaker': 0.25, 'feeMaker': 0.15}, 'japonicus/ja354DEMARSI': {'CCI': {'consistant': 20.020000000000003, 'active': 0.32, 'history': 65.7, 'thresholds': {'up': 148.0, 'down': -91.0, 'persistence': 8.86}}, 'LRC': {'active': 0.18, 'thresholds': {'up': 30.0, 'down': -16.0}, 'depth': 9.45}, 'SMMA': {'weight': 11.95, 'thresholds': {'up': 0.031, 'down': -0.082}, 'active': 0.48}, 'TSI': {'active': 0.09, 'thresholds': {'up': 32.6, 'down': -24.2}, 'short': 7.41, 'long': 17.6}, 'DEMA': {'long': 16.83, 'thresholds': {'up': 0.047, 'down': -0.069}, 'short': 12.76, 'active': 0.54}, 'PPO': {'active': 0.19, 'signal': 13.07, 'thresholds': {'up': 0.165, 'down': -0.435}, 'short': 13.2, 'long': 18.2}, 'RSI': {'active': 0.88, 'thresholds': {'up': 51.65, 'down': 41.099999999999994}, 'interval': 18.200000000000003}, 'persistence': 1.04}, 'backtest': {'daterange': {'from': '2018-01-18 09:20:07', 'to': '2018-01-19 09:20:07'}}}}
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/urllib3/connection.py", line 141, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/usr/local/lib/python3.4/dist-packages/urllib3/util/connection.py", line 83, in create_connection
    raise err
  File "/usr/local/lib/python3.4/dist-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.4/dist-packages/urllib3/connectionpool.py", line 357, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.4/http/client.py", line 1090, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.4/http/client.py", line 1128, in _send_request
    self.endheaders(body)
  File "/usr/lib/python3.4/http/client.py", line 1086, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python3.4/http/client.py", line 924, in _send_output
    self.send(msg)
  File "/usr/lib/python3.4/http/client.py", line 859, in send
    self.connect()
  File "/usr/local/lib/python3.4/dist-packages/urllib3/connection.py", line 166, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.4/dist-packages/urllib3/connection.py", line 150, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0xaeba1f2c>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/usr/local/lib/python3.4/dist-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/local/lib/python3.4/dist-packages/urllib3/util/retry.py", line 388, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=3000): Max retries exceeded with url: /api/backtest (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xaeba1f2c>: Failed to establish a new connection: [Errno 111] Connection refused',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "japonicus.py", line 99, in <module>
    gekko_generations(TargetParameters, GenerationMethod, EvaluationMode, web=APP)
  File "/var/www/japonicus/evolution_generations.py", line 109, in gekko_generations
    resultInterface.showResults(World)
  File "/var/www/japonicus/resultInterface.py", line 39, in showResults
    ValidationDataset)
  File "/var/www/japonicus/resultInterface.py", line 64, in stratSettingsProofOfViability
    (q, s), m = World.tools.Evaluate([W], Individual, 'http://localhost:3000')
  File "/var/www/japonicus/evolution_generations.py", line 30, in aEvaluate
    DateRange, phenotype, gekkoUrl)
  File "/var/www/japonicus/promoterz/evaluation/gekko.py", line 268, in Evaluate
    Debug=genconf.gekkoDebug) for DR in DateRange ]
  File "/var/www/japonicus/promoterz/evaluation/gekko.py", line 268, in <listcomp>
    Debug=genconf.gekkoDebug) for DR in DateRange ]
  File "/var/www/japonicus/promoterz/evaluation/gekko.py", line 112, in runBacktest
    result = httpPost(url, gekko_config)
  File "/var/www/japonicus/promoterz/evaluation/gekko.py", line 41, in httpPost
    raise e
  File "/var/www/japonicus/promoterz/evaluation/gekko.py", line 31, in httpPost
    Request = requests.post(URL, json=data)
  File "/usr/local/lib/python3.4/dist-packages/requests/api.py", line 112, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.4/dist-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/requests/adapters.py", line 508, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=3000): Max retries exceeded with url: /api/backtest (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xaeba1f2c>: Failed to establish a new connection: [Errno 111] Connection refused',))

GEKKO OUTPUT : 

2018-01-22 23:39:18 (INFO): Setting up:
2018-01-22 23:39:18 (INFO):      Paper Trader
2018-01-22 23:39:18 (INFO):      Paper trader that simulates fake trades.
2018-01-22 23:39:18 (INFO):

2018-01-22 23:39:18 (INFO): Setting up:
2018-01-22 23:39:18 (INFO):      Performance Analyzer
2018-01-22 23:39:18 (INFO):      Analyzes performances of trades
2018-01-22 23:39:18 (INFO):

2018-01-22 23:39:18 (INFO):

2018-01-22 23:39:18 (INFO): Setting up:
2018-01-22 23:39:18 (INFO):      Paper Trader
2018-01-22 23:39:18 (INFO):      Paper trader that simulates fake trades.
2018-01-22 23:39:18 (INFO):

2018-01-22 23:39:18 (INFO): Setting up:
2018-01-22 23:39:18 (INFO):

2018-01-22 23:39:18 (INFO):      Paper Trader
2018-01-22 23:39:18 (INFO):      Paper trader that simulates fake trades.
2018-01-22 23:39:18 (INFO): Setting up:
2018-01-22 23:39:18 (INFO):      Performance Analyzer
2018-01-22 23:39:18 (INFO):      Analyzes performances of trades
2018-01-22 23:39:18 (INFO):

2018-01-22 23:39:18 (INFO): Setting up:
2018-01-22 23:39:18 (INFO):      Performance Analyzer
2018-01-22 23:39:18 (INFO):      Analyzes performances of trades
2018-01-22 23:39:18 (INFO):

2018-01-22 23:39:18 (INFO):

2018-01-22 23:39:18 (INFO):

2018-01-22 23:39:18 (INFO): Setting up:
2018-01-22 23:39:18 (INFO):      Paper Trader
2018-01-22 23:39:18 (INFO):      Paper trader that simulates fake trades.
2018-01-22 23:39:18 (INFO):

2018-01-22 23:39:18 (INFO): Setting up:
2018-01-22 23:39:18 (INFO):      Performance Analyzer
2018-01-22 23:39:18 (INFO):      Analyzes performances of trades
2018-01-22 23:39:18 (INFO):

2018-01-22 23:39:18 (INFO): Setting up:
2018-01-22 23:39:18 (INFO):      Paper Trader
2018-01-22 23:39:18 (INFO):      Paper trader that simulates fake trades.
2018-01-22 23:39:18 (INFO):

2018-01-22 23:39:18 (INFO):

2018-01-22 23:39:18 (INFO): Setting up:
2018-01-22 23:39:18 (INFO):      Performance Analyzer
2018-01-22 23:39:18 (INFO):      Analyzes performances of trades
2018-01-22 23:39:18 (INFO):

2018-01-22 23:39:18 (INFO):     WARNING: BACKTESTING FEATURE NEEDS PROPER TESTING
2018-01-22 23:39:18 (INFO):     WARNING: ACT ON THESE NUMBERS AT YOUR OWN RISK!
2018-01-22 23:39:19 (INFO):     WARNING: BACKTESTING FEATURE NEEDS PROPER TESTING
2018-01-22 23:39:19 (INFO):     WARNING: ACT ON THESE NUMBERS AT YOUR OWN RISK!
2018-01-22 23:39:19 (INFO):     WARNING: BACKTESTING FEATURE NEEDS PROPER TESTING
2018-01-22 23:39:19 (INFO):     WARNING: ACT ON THESE NUMBERS AT YOUR OWN RISK!
2018-01-22 23:39:19 (INFO):     WARNING: BACKTESTING FEATURE NEEDS PROPER TESTING
2018-01-22 23:39:19 (INFO):     WARNING: ACT ON THESE NUMBERS AT YOUR OWN RISK!
2018-01-22 23:39:19 (INFO):     WARNING: BACKTESTING FEATURE NEEDS PROPER TESTING
2018-01-22 23:39:19 (INFO):     WARNING: ACT ON THESE NUMBERS AT YOUR OWN RISK!
  --> POST /api/backtest 200 4,755ms 17.08kb
  --> POST /api/backtest 200 4,771ms 17.08kb
  --> POST /api/backtest 200 4,765ms 17.08kb
  --> POST /api/backtest 200 4,704ms 17.08kb
  --> POST /api/backtest 200 4,706ms 17.08kb
OterLabb commented 6 years ago

Getting this exact same error. Ubuntu 16.04, newest gekko and japonicus as of today. Python 3.5.2 Ip set to 192.168.1.110 instead of localhost.

Gab0 commented 6 years ago

@OterLabb So latest gekko runs @192.168.1.110? I'm checking japonicus, maybe at some place the address is redefined beyond Settings, in a nice bug.

@grogcw It seems it all works fine until that random evaluation that happens @EPOCH 20 on std settings xD I'm fixing everything.. update should be online tonite. Thanks for the report!

OterLabb commented 6 years ago

Yes gekko runs @192.168.1.110, its starting to run, but crashes after a while, 5-10 minutes maybe.

grogcw commented 6 years ago

@Gab0 Ok, I'm waiting for the update.

By the way, if it is relevant to you, here's another log about the crash, it right after EPOCH 1.

EPOCH 1 &11
Maximum profit -4.542   Average profit -6.241
Minimum profit -11.431  Profit variation 2.093
Population size 42  Max population size 30
Avg trade number 0.705  Avg sharpe ratio 0

Elder dies 0

Epoch runs in 57.03 seconds;

Selecting 1+4 individues, random test;
4 selected;
1 selected;
Error: config failure
http://localhost:3000/api/backtest

Here's my conf file (relevant parameters) :

        'Global': {
            'gekkoPath': '/var/www/gekko',
            'configFilename': 'DEMA.js',
            'save_dir': "output",
            'log_name': 'evolution_gen.csv',
            'RemoteAWS': '../AmazonSetup/hosts',
            'GekkoURLs': [ 'http://192.168.0.62:3000' ],
            'GekkoInstanceUrl': [ 'http://192.168.0.62:3000' ]
        },
        # genetic algorithm settings
        'generations': {
            'gekkoDebug': True,
            'showIndividualEvaluationInfo': False,
            'POP_SIZE': 30,
            'NBEPOCH': 19,
            'evaluateSettingsPeriodically': 2,
            'deltaDays': 1,
            'NBCandlestickData': 4,

            'cxpb': 0.3,
            'mutpb': 0.7,
            '_lambda': 7,

Thanks for taking care :)

Gab0 commented 6 years ago

Fixed ;D