DeviaVir / zenbot

Zenbot is a command-line cryptocurrency trading bot using Node.js and MongoDB.
MIT License
8.22k stars 2.04k forks source link

Implement genetic algo for fine tuning strategy parameters #298

Closed gelotus closed 6 years ago

gelotus commented 7 years ago

Take a look genetic-js It is also present on npm. Let's discuss here

arpheno commented 7 years ago

hurr hurr hurr, i'm waaaay ahead of you. Unfortunately it's in python but i'm getting some good results #299

I have very little documentation currently, unfortunately.

Inside zen/ there is a python file main that will launch a genetic algorithm to train the parameters of a given strategy.

To launch:

docker-compose up
docker-compose exec server bash
fab backfill_local:<days>  
cd zen
python -m scoop main.py <product> <days> <individuals> <strategy>

product can be either ETH-BTC or BTC-CUR where CUR stands for "currency"

image

I also wrote a thesis on parameter tuning with genetic algorithms, if you're interested: Automatic Audio Encoder Tuning Using Evolutionary Algorithms.pdf

ch0wdan commented 7 years ago

@arpheno How are you using your python strat with a JS based system? Does your python algo run theny ou take the outputted data over to zenbot?

arpheno commented 7 years ago

Yes.

On Fri, Jun 16, 2017 at 6:57 PM Jordan notifications@github.com wrote:

@arpheno https://github.com/arpheno How are you using your python strat with a JS based system? Does your python algo run theny ou take the outputted data over to zenbot?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/carlos8f/zenbot/issues/298#issuecomment-309078942, or mute the thread https://github.com/notifications/unsubscribe-auth/ABBzGh2FMd3a-oRbJMK2MYmQ0I2UUyeaks5sErQHgaJpZM4N8Xyg .

gelotus commented 7 years ago

@arpheno Great work, I'll take a look. Have you seen the algo that I have linked?

gelotus commented 7 years ago

I've setting up your work. I'm still waiting for backfilling.. Some suggestion, can you make it agnostic in environment and selector? With a conf file? I've see that there is much stuff hardcoded, like specific commands for docker. And in zen/tests/test_integration.py this hardcoded selector main('gdax.BTC-ETH',120)

gelotus commented 7 years ago

Finally i've succeeded in run your genetics.. After a fight with the new f-strings feature in python 3.6 and after.. I'm go to run some simulations and we will see

arpheno commented 7 years ago

I will be glad to hear how it went. Currently 4 people are running it including you and me.

On Sun, Jun 18, 2017 at 11:49 PM gelotus notifications@github.com wrote:

Finally i've succeeded in run your genetics.. After a fight with the new f-strings feature in python 3.6 and after.. I'm go to run some simulations and we will see

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/carlos8f/zenbot/issues/298#issuecomment-309305251, or mute the thread https://github.com/notifications/unsubscribe-auth/ABBzGhTrr80VtSOnyldDRc19d8gmKWcxks5sFZtxgaJpZM4N8Xyg .

gelotus commented 7 years ago

@arpheno sorry, i'm not expert in python, but seems to be a bug in evaluate_zen() that not pass the right values. If I print fitness var I obtain always [-100, -100]. Please take a look

gelotus commented 7 years ago

Ok, with your fix all seems to work right. Let's show some results: Generation 181 {'avg': 1.1251920040357279, 'std': 6.1372615421965255, 'min': -15.320000000000269, 'max': 5.287425556773063, 'len': 20}

Current Hall of Fame:
../zenbot.sh sim gdax.BTC-GBP --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 Randal Pitts
../zenbot.sh sim gdax.BTC-USD --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 Stephanie Houchins
../zenbot.sh sim gdax.BTC-USD --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 Patrick Smith
../zenbot.sh sim gdax.BTC-GBP --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 Thomas Tomlin
../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 James Alvarez
../zenbot.sh sim gdax.BTC-GBP --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 Cody Harvey
../zenbot.sh sim gdax.BTC-GBP --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 Jason Dixon
../zenbot.sh sim gdax.BTC-GBP --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 Lynn Hetrick
../zenbot.sh sim gdax.BTC-GBP --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 Tamika Vega
../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 Jill Ogrady
../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 Patrica Espinoza
../zenbot.sh sim gdax.BTC-GBP --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 Harold Richardson
../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 Shirley Harten
../zenbot.sh sim gdax.BTC-USD --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.5633 --period 95m  [5.62, 6.55] 5.287425556773063 Dustin Wright
../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 James Mckinney
4 Elites will survive, they're currently the strongest:
../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 Randal Pitts
../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 Lynn Hetrick
../zenbot.sh sim gdax.BTC-USD --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 Tamika Vega
../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 Stephanie Houchins
8 Specialists will survive, they're the best in their domain:
../zenbot.sh sim gdax.BTC-USD --strategy sar --avg_slippage_pct 0.4 --min_periods 10 --sar_af 0.0064 --sar_max_af 0.0778 --period 131m  [-7.91, 13.4] -7.910000000556141 Celina Belzer
../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0072 --sar_max_af 0.2336 --period 94m  [6.5, 0.06] 0.058404866221893226 Scott Stancil
../zenbot.sh sim gdax.BTC-USD --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0072 --sar_max_af 0.2336 --period 94m  [6.5, -0.21] -0.21121792215630394 Felix Roy
../zenbot.sh sim gdax.BTC-USD --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0072 --sar_max_af 0.2336 --period 94m  [6.5, 0.06] 0.058404866221893226 John Cook
../zenbot.sh sim gdax.BTC-USD --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0072 --sar_max_af 0.2336 --period 94m  [6.5, -0.56] -0.5608584095551045 Harriet Martin
../zenbot.sh sim gdax.BTC-USD --strategy sar --avg_slippage_pct 0.4 --min_periods 4 --sar_af 0.0078 --sar_max_af 0.2336 --period 78m  [-8.53, 13.19] -8.530000000369082 Anita Farnsworth
../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 4 --sar_af 0.0091 --sar_max_af 0.2336 --period 78m  [-15.32, 13.63] -15.320000000000269 Jo Cleveland
../zenbot.sh sim gdax.BTC-GBP --strategy sar --avg_slippage_pct 0.4 --min_periods 4 --sar_af 0.0078 --sar_max_af 0.2336 --period 78m  [-8.53, 13.19] -8.530000000369082 Juanita Hurst
Some other have fought their way through:
../zenbot.sh sim gdax.BTC-GBP --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 Lynn Hetrick
../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 Randal Pitts
../zenbot.sh sim gdax.BTC-USD --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 Lynn Hetrick
../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 Lynn Hetrick
../zenbot.sh sim gdax.BTC-EUR --strategy sar --avg_slippage_pct 0.4 --min_periods 13 --sar_af 0.0091 --sar_max_af 0.2336 --period 95m  [5.62, 6.55] 5.287425556773063 Lynn Hetrick
...

With long simulations the simulations directory grows fast in size, needs a fix. In the hall of fame all the population have the same parameters.

arpheno commented 7 years ago

can you pull the latest version? I think both issues should be addressed there. I made it so one individual can only ever live once ( with the same parameters).

gelotus commented 7 years ago

Well done, whit your last commit all work seamlessly. But.. (philosophical thinking) it works on static data. What happens if the strong elites dies after n generation and the datas are refreshed with an auto backfill over the same n generations? I think that if one keep the sim running in the same time that the trading bot working we will have a fresh and plus or minus the right parameters for actual period. There be also possible to create an automated bot refresh procedure that pass the new parameters automatically. What do you think?

gelotus commented 7 years ago

hey swozny what's happen in your repo? Anyway I've tried to merge some of my ideas in your work. Environment configuration, selectors configuration and autobackfilling. The draw function is broken but i have create a little 'if' that point in conf for turning on or off. I've also setting minimum limit in parameters. Now i'm working in linux env, need some good guy that try this in docker, because i'm the king of the ignorants on this env. Will be a good thing to append a "date-time from to" to the hall of fame for improve analisys of results with autobackfill turned on.

If someone want to try this is my repo: https://github.com/gelotus/zenbot

Take a look in conf.py, command syntax varied a little bit: for manual backfilling <zenbot_dir>$ fab autobackfill_local:"<selector>,<days>" for starting sim <zenbot_dir>/zen $ python3.6 -m scoop main.py <selector> <days> [<population>][<strategy>] [backfill]

Selectors are in conf, days are the days of trade, polulation in the number of live individuals after a generation change, min 10 default 10, strategy is the same of zenbot, default trend_ema, if backfill is present, it will turn on autobackfill timer during sim for updating the trade history.

Note that in my repo the docker files are absent. Pick those in carlos repo

arpheno commented 7 years ago

I tried to merge your PR and it deleted the dockerfiles. I got a lot of complaints from other people when I merged it, so I had to revert :(

On Thu, Jun 29, 2017 at 12:11 AM gelotus notifications@github.com wrote:

hey swozny what's happen in your repo? Anyway I've tried to merge some of my ideas in your work. Environment configuration, selectors configuration and autobackfilling. The draw function is broken but i have create a little 'if' that point in conf for turning on or off. I've also setting minimum limit in parameters. Now i'm working in linux env, need some good guy that try this in docker, because i'm the king of the ignorants on this env. Will be a good thing to append a "date-time from to" to the hall of fame for improve analisys of results with autobackfill turned on.

If someone want to try this is my repo: https://github.com/gelotus/zenbot

Take a look in conf.py, command syntax varied a little bit: for manual backfilling $ fab autobackfill_local:"," for starting sim

/zen $ python3.6 -m scoop main.py [][] [backfill] Selectors are in conf, days are the days of trade, polulation in the number of live individuals after a generation change, min 10 default 10, strategy is the same of zenbot, default trend_ema, if backfill is present, it will turn on autobackfill timer during sim for updating the trade history. Note that in my repo the docker files are absent. Pick those in carlos repo — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub , or mute the thread .
gelotus commented 7 years ago

ok, let's try to implement a windows+docker env and test with it..

arpheno commented 7 years ago

I think I'll get to work on this on the weekend :) I'm absolutely amazed that someone is building on my work, not gonna let it go to waste.

On Thu, Jun 29, 2017 at 9:13 PM gelotus notifications@github.com wrote:

ok, let's try to implement a windows+docker env and test with it..

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/carlos8f/zenbot/issues/298#issuecomment-312065191, or mute the thread https://github.com/notifications/unsubscribe-auth/ABBzGo4P6BMB1gmgH-DupmY24cGNmCorks5sI_GegaJpZM4N8Xyg .

Maximus-CZ commented 7 years ago

Just droppin my 2 cents: would be cool if trading would be performed by agent with best score, while in the background there is still evolution going on data of past day or so (quickly iterate) and if found better agent, use that. (While keeping backups in case of unpredicted disasters)

arpheno commented 7 years ago

I think you're very courageous if you're not double checking the strategy the optimizer comes up with before throwing it at real money.

On Thu, Jun 29, 2017 at 11:02 PM Maximus-CZ notifications@github.com wrote:

Just droppin my 2 cents: would be cool if trading would be performed by agent with best score, while in the background there is still evolution going on data of past day or so (quickly iterate) and if found better agent, use that. (While keeping backups in case of unpredicted disasters)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/carlos8f/zenbot/issues/298#issuecomment-312103202, or mute the thread https://github.com/notifications/unsubscribe-auth/ABBzGqFtuWJh5i04MVIt0Wl29exXNHcUks5sJA2PgaJpZM4N8Xyg .

Maximus-CZ commented 7 years ago

Well before agent would trade with real money, it would first need to run trough simulations to verify its profitability, If I had 2 agents with simulated +10% and +30% ROI respectively, I wouldnt really see a problem switching to better agent at runtime. On top of this, having Zenbot run several parallel agents in papertrading mode can also provide indication of which agent would be best choice for given market/day.

gelotus commented 7 years ago

A little question.. Who uses docker? And on windows or on linux? I've see that the last docker for windows works only on windows 10, both stable and edge versions. For windows 7-8-8.1 there is docker-toolbox. That's very fragmented, on linux i don't see the convenience of using docker.. But i don't know, need an advice

gelotus commented 7 years ago

@arpheno are you vanished? :) Take a look in my repo, docker was reinserted.

l-j-g commented 7 years ago

Hey Guys, Just want to say thanks for all your efforts to implement genetic algorithms into zenbot. It has been the first introduction to the topic for me and i think im hooked :+1:

For the last few weeks I have been trying to add covariance matrix adaptation evolution strategy, which i believe will automatically adjust cx and mut probabilities to help with convergence and to prevent getting stuck on local maximums allowing us to develop and optimise more complex strategies.

I think this should be possible and relatively easy using the tools from the python deap package a la:

http://deap.readthedocs.io/en/master/examples/cmaes.html https://github.com/DEAP/deap/blob/a5d77d1e30f7674948bb6a2c4381fa4e247ac89d/examples/es/fctmin.py

Unfortunately this is my first introduction to both python and evolution strategies, and to be honest, i'm a bit out of my depth. @gelotus, @arpheno I was wondering if you guys, (or anyone else) would be able to have a look (im sad to see development seems to have stopped on this fork). I think this would be very beneficial to zenbot development and for me personally very rewarding (emotionally... maybe financially too !) =)

So far I have basically tried to frankenstein the example into the code by swonzy but it is not working with the compiling error:

TypeError: sequence item 0: expected str instance, Individual found

I think I need help to correctly register fitness and individual functions into the deap toolbox..


import numpy

from functools import partial
from timer import ThreadTimer
from deap import algorithms, base, benchmarks, cma, creator, tools
from deap.tools import cxTwoPoint, mutGaussian
from scoop import shared
from blessings import Terminal
from subprocess import run, DEVNULL
from conf import indpb, sigma, partitions, selectors, path, bkfint
from evaluation import evaluate_zen, Andividual
from evolution import evolve

term = Terminal()

creator.create("FitnessMax", base.Fitness, weights=(1.0,))
creator.create("Individual", list, fitness=creator.FitnessMax)

toolbox = base.Toolbox()

N=30

def main(instrument, days, popsize, zstrategy='trend_ema', nobf=None):
    print(term.blue("Starting evolution...."))
    evaluate = partial(evaluate_zen, days=days)
    toolbox.register("evaluate", evaluate_zen, days=days)
    print(term.blue("Evaluating ") + term.green(str(popsize)) + term.blue(" individuals over ")
    + term.green(str(days)) + term.blue(" days in ") + term.green(str(partitions)) + term.blue(" partitions."))
    Andividual.path = path
    Andividual.instruments = selectors[instrument]
    Andividual.mate = cxTwoPoint
    Andividual.mutate = partial(mutGaussian, mu=0, sigma=sigma, indpb=indpb)
    Andividual.strategy = zstrategy
    print(term.blue("Mating function is ") + term.green(str(Andividual.mate)))
    print(term.blue("Mutating function is ") + term.green(str(Andividual.mutate)))

    numpy.random.seed(128)

    strategy = cma.Strategy(centroid=[5.0]*N, sigma=5.0, lambda_=20*N)
    toolbox.register("generate", strategy.generate, creator.Individual)
    toolbox.register("update", strategy.update)

    hof = tools.HallOfFame(1)
    stats = tools.Statistics(lambda ind: ind.fitness.values)
    stats.register("avg", numpy.mean)
    stats.register("std", numpy.std)
    stats.register("min", numpy.min)
    stats.register("max", numpy.max)

    algorithms.eaGenerateUpdate(toolbox, ngen=250, stats=stats, halloffame=hof)

    res = evolve(evaluate, Andividual, popsize)
    return res, hof[0].fitness.values[0]```
jaketblank commented 6 years ago

I too am interested in learning more about live ML/genetic algo integration for realtime trade-bot strategy optimization, but my knowledge here is limited at the moment. I'm more than happy to start digging into the code myself, and probably will, but am also curious if @gelotus and/or @arpheno have any news regarding their efforts 🔥

brucetus commented 6 years ago

Has anybody encountered this problem? Im using a custom strategy and the only option it has is a --period option. It works fine for regular simulations and trading. I'm getting this error:


It's breeding season, we're expecting new members of the tribe...
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.6/site-packages/scoop/bootstrap/__main__.py", line 302, in <module>
    b.main()
  File "/usr/local/lib/python3.6/site-packages/scoop/bootstrap/__main__.py", line 92, in main
    self.run()
  File "/usr/local/lib/python3.6/site-packages/scoop/bootstrap/__main__.py", line 290, in run
    futures_startup()
  File "/usr/local/lib/python3.6/site-packages/scoop/bootstrap/__main__.py", line 271, in futures_startup
    run_name="__main__"
  File "/usr/local/lib/python3.6/site-packages/scoop/futures.py", line 64, in _startup
    result = _controller.switch(rootFuture, *args, **kargs)
  File "/usr/local/lib/python3.6/site-packages/scoop/_control.py", line 253, in runController
    raise future.exceptionValue
  File "/usr/local/lib/python3.6/site-packages/scoop/_control.py", line 127, in runFuture
    future.resultValue = future.callable(*future.args, **future.kargs)
  File "/usr/local/lib/python3.6/runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "/usr/local/lib/python3.6/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "main.py", line 68, in <module>
    res = main(INSTRUMENT, TOTAL_DAYS, popsize, strategy)
  File "main.py", line 26, in main
    res = evolve(evaluate, Andividual, popsize)
  File "/app/zen/evolution/__init__.py", line 18, in evolve
    return algorithm(cls, popsize, futures.map, evaluate, select, breed, mutate, stats, history, hof)
  File "/app/zen/evolution/core.py", line 22, in algorithm
    offspring = breed(population)
  File "/app/zen/evolution/core.py", line 43, in breed
    child1, child2 = parent1 + parent2
  File "/app/zen/evolution/individual_base.py", line 36, in __add__
    child1, child2 = self.__class__.mate(*couple)
  File "/usr/local/lib/python3.6/site-packages/deap/tools/crossover.py", line 44, in cxTwoPoint
    cxpoint2 = random.randint(1, size - 1)
  File "/usr/local/lib/python3.6/random.py", line 220, in randint
    return self.randrange(a, b+1)
  **File "/usr/local/lib/python3.6/random.py", line 198, in randrange
    raise ValueError("empty range for randrange() (%d,%d, %d)" % (istart, istop, width))
ValueError: empty range for randrange() (1,1, 0)**```
jaketblank commented 6 years ago

The randrange() function is being called on a range with no depth (1,1), hence the error.

Looking up the trace, we see the function is calling randrange(a, b+1). On mobile right now so debugging is limited, but I would check to see what variable "b" is supposed to represent, and figure out why it's being passed through as 0

brucetus commented 6 years ago

@jaketblank I can't find the random.py file in my computer. I'm running on Docker. Where would I be able to read random.py? Also I've been able to use zen on other strategies like trend_ema so I don't think that zen itself is broken.

jaketblank commented 6 years ago

The problem isn't with the randrange() function - if it was there would be a major problem with all python code that uses it :)

(That file should be located in "/usr/local/lib/python3.6/" on your system, just for reference)

I'd start looking in "individual_base.py" and look at the "mate()" method. See, then track forwards to see where values are being generated/sent to randrange()

brucetus commented 6 years ago

@jaketblank

random.randint(1, size - 1)
self.randrange(a, b+1)
randrange(1,1, 0)

that means b = 0 and if b = size - 1 then size = 1 From crossover.py: size = min(len(ind1), len(ind2)) It's the minimum length of the two individuals being crossed over Wouldn't that mean that at least one of the lengths of the individuals being crossed over is 1? I'm not sure what that means but I think the problem is that the strategy I'm using only has one option: period size. I think this makes the length of the individuals too similar, but I'm not sure. I think a strategy with one option would be better optimized using the backtester script. Thanks though.

albabosh commented 6 years ago

I get a strange error:

python3.6 -m scoop main.py BTC-CUR 10 10 macd

When genetic_algo starts it prints:

Starting evolution.... Evaluating 10 individuals over 10 days in 2 partitions. Mating function is <function cxTwoPoint at 0x7f5ee6defa60> Mutating function is functools.partial(<function mutGaussian at 0x7f5ee1ae0c80>, mu=0, sigma=20, indpb=0.3) Sampling an initial valid population, this may take a while... Currently 0 valid individuals

Then after a while: xxxxxxxxxxCurrently 1 valid individuals

The algo never stops, repeatedly printing: xxxxxxxxxxCurrently 1 valid individuals

What could be wrong? The DB was correctly backfilled.

albabosh commented 6 years ago

Should I worry about this thing:

/usr/local/lib/python3.6/dist-packages/deap/tools/_hypervolume/pyhv.py:33: ImportWarning: Falling back to the python version of hypervolume module. Expect this to be very slow. "module. Expect this to be very slow.", ImportWarning)

enricocas commented 6 years ago

@albabosh I'm having the same output of you xxxxxxxxxxCurrently 1 valid individuals for each cycle.

Heerpa commented 6 years ago

Have a look at pull request #1078. Now this issue doesn't come up any more for me (because a set can hold more than 1 individual now, after deleting the overwritten version of eq .

LoneWolf345 commented 6 years ago

Can this be closed?

enricocas commented 6 years ago

For me it's ok

Il 06 mar 2018 10:13 PM, "LoneWolf345" notifications@github.com ha scritto:

Can this be closed?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/DeviaVir/zenbot/issues/298#issuecomment-370930228, or mute the thread https://github.com/notifications/unsubscribe-auth/AA9DL5I96Va_CsxDMxZyyaYjLvE8UZsrks5tbvwMgaJpZM4N8Xyg .