Genetic optimization

[chrisdk2015]

Created this issue for tracking the genetic optimization

https://github.com/chrisdk2015/LeanOptimization

It uses the GAF framework and is currently for running on the command line.

Howto parallize this would be nice to know. Maybe one needs to change the GAF framework.

[bizcad]

@chrisdk2015

I got the genetic algorithm running without blurring by on the console. However, after 1723 trials, it blows up my Visual Studio. I mean VS just quits with no notice or anything. It has happened twice, once with console blur and once without. It seems to happen in the same place but I cannot catch an Exception.

I was wondering how many trials were needed to resolve the genetic algorithm. And also did your crash like that. Of course, if you were waiting 40 seconds between trials, it would have taken all night. My current version did the 1723 trials in 1:09:22 or 2-3 seconds per trial and only shows the 1 line per trial in the console.

I copied ConsoleResultsHandler and modified it into OptimizationResutsHandler. I also commented out line 62 in JobQueue because it was Log.Trace("dodah") before the Log.Handler was set to a FileLogHandler.

Here is a link to a dropbox folder with the log.txt and log.csv files for the 1723 trials. https://www.dropbox.com/sh/zm19bgmuo1t5os0/AACt4pHp6zbE1KNmWxnhlO7wa?dl=0

I am pushing the complete project to https://github.com/bizcad/LeanOpto

On line 89 of Program, I set

            Log.LogHandler = (ILogHandler)new FileLogHandler();
            Log.DebuggingEnabled = true;
            Log.DebuggingLevel = 2;

so that Log output is only sent to a file. Then I replace all the Console.WriteLines (except for 2) to Log.Trace, Debug or Error.

I also commented the parts that refer to writing chart files. We do not need them.

Nick

[chrisdk2015]

@bizcad

I think there is a memory leak somewhere.

At least here when I run start running it the memory usage keeps going up.

Maybe you can put the memory usage in the log file to see if that is the reason.

I tried to do a profiling here with mono but I just get segfaults so it is of no help. Maybe you could profile it in VS? I think the profile there is better.

With the OptimizationResultHandler it goes a bit faster.

For the termination criteria you could use the following

private static double oldFitness = Double.NaN;

public static bool Terminate(Population population, 
            int currentGeneration, long currentEvaluation)
        {
            var fittest = population.GetTop(1)[0];
            if (oldFitness != Double.NaN) {
                var diff = System.Math.Abs (fittest.Fitness - oldFitness);
                if (diff < 1E-02)
                    return true;
            }
            oldFitness = fittest.Fitness;
            return false;
            //return currentGeneration > 10;
        }

instead of using the generation number to stop.

Also to avoid running that many times the number of chromosomes and genes per chromosome could be lowered

//create the chromosomes
            for (var p = 0; p < 10; p++)
            {

                var chromosome = new Chromosome();
                for (int i = 0; i < 10; i++) {
                    ConfigVars v = new ConfigVars ();
                    v.vars ["EMA_VAR1"] = RandomNumberBetweenInt (0, 20);
                    v.vars ["EMA_VAR2"] = RandomNumberBetweenInt (0, 100);

                    chromosome.Genes.Add (new Gene (v));
                }
                chromosome.Genes.ShuffleFast();
                population.Solutions.Add(chromosome);
            }

The question is how big the default population should be? I don't know.

By the way the code in my github has a solution for multiple variables using the ConfigVars class.

[bizcad]

@chrisdk2015 I am not experienced with the Microsoft profiler, but I tried running it. The data file filled up what was left of my C:\ drive. :o( I deleted the file and ran again stopping after about 200 iterations. The resulting report did not tell me anything useful. I dumped the profiler and ran the Program.cs.

I watched the Task manager for memory usage and it did not grow for 2.5 hours. So I do not believe there is a memory leak.

I put in a stop at interation 1721 to try to catch the error, but I formed the message incorrectly: string.Format("{1}", iterationnumber); A really stupid mistake. I changed the {1} to {0}. Now I need to run it again to get to the stop point. It is taking about 2.5 seconds per iteration so it is a 2.5 hour wait.

After I get there I will try integrating your new code. It will probably be tomorrow before I have something.

Nick

[gururise]

Been trying to get this running the past two days, I keep crashing on line 356 in Program.cs: var res = (double)rc.Run (val); On the terminal, I get this error:

20151103 08:52:34 ERROR:: Loader.TryCreateILAlgorithm(2): Sequence contains no matching element 20151103 08:52:34 Trace:: Initializing... 20151103 08:52:34 ERROR:: Engine.Main(): Error running algorithm: Object reference not set to an instance of an object >> at QuantConnect.Lean.Engine.Results.ConsoleResultHandler.RuntimeError (System.String message, System.String stacktrace) [0x0000f] in /home/gene/Downloads/Lean/Engine/Results/ConsoleResultHandler.cs:248 at QuantConnect.Lean.Engine.Engine.Run (QuantConnect.Packets.AlgorithmNodePacket job, System.String assemblyPath) [0x00305] in /home/gene/Downloads/Lean/Engine/Engine.cs:249

In my IDE (mono-develop), I get:

System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.

[mchandschuh]

20151103 08:52:34 ERROR:: Loader.TryCreateILAlgorithm(2): Sequence contains no matching element

This message sounds like the Loader was unable to find the algorithm specified in configuration.

The function _multipleTypeNameResolverFunction gets invoked with all type names resolved from the assembly to select a single algorithm type to instantiate. In many places in the code, the Loader is initialized to force exactly one match on the type name via the LINQ .Single() call.

Check config.json 'algorithm-type-name' matches properly. Also, try stepping through the invocation of the '_multipleTypeNameResolverFunction' function from within the TryCreateILAlgorithm method in the Loader class.

-Michael

On Tue, Nov 3, 2015 at 11:59 AM, gururise notifications@github.com wrote:

Been trying to get this running the past two days, I keep crashing on line 356 in Program.cs: var res = (double)rc.Run (val); On the terminal, I get this error:

20151103 08:52:34 ERROR:: Loader.TryCreateILAlgorithm(2): Sequence contains no matching element 20151103 08:52:34 Trace:: Initializing... 20151103 08:52:34 ERROR:: Engine.Main(): Error running algorithm: Object reference not set to an instance of an object >> at QuantConnect.Lean.Engine.Results.ConsoleResultHandler.RuntimeError (System.String message, System.String stacktrace) [0x0000f] in /home/gene/Downloads/Lean/Engine/Results/ConsoleResultHandler.cs:248 at QuantConnect.Lean.Engine.Engine.Run (QuantConnect.Packets.AlgorithmNodePacket job, System.String assemblyPath) [0x00305] in /home/gene/Downloads/Lean/Engine/Engine.cs:249

In my IDE (mono-develop), I get:

System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.

— Reply to this email directly or view it on GitHub https://github.com/QuantConnect/Lean/issues/190#issuecomment-153417254.

[bizcad]

@gururise If you copied the code from @chrisdk2015 Optimization project on github, my guess is your problem is on line 154 of Program.cs

154 _resultshandler = new DesktopResultHandler ();

Try changing all the DesktopResultHandler to ConsoleResultHandler. DesktopResultHandler expects a Desktop application to write results to. In the console application there is no desktop.

Also take a look at my take on a complete solution including Lean and all it's sub-projects. This link is also in my reply above. I changed DesktopResultHandler in three places to an OptimizationResultHandler with some commented lines next for ConsoleResultHandler. If you want to see the results flash by in the console comment out the OptimizationResultsHandler and uncomment the ConsoleResultsHandler.

The code for the OptimizationResultsHandler is in the Engine\Results folder and is based upon ConsoleResultsHandler.

I run in Windows/VisualStudio, so I place a breakpoint about line 75 of Program.cs at the top of the LaunchLean function and step from there to find my error.

            Config.Set("environment", "backtesting");
            string algorithm = "EMATest";       // put this algorithm is QuantConnect.Algorithm.CSharp folder
            Config.Set("algorithm-type-name", algorithm);

I hope this helps.

Bizcad aka Nick Stein

[jameschch]

This is a fairly old ticket and it appears work has stalled. I have recently overhauled the LeanOptimization project here:

https://github.com/jameschch/LeanOptimization

Anyone still interested in this functionality please feel free to provide feedback, open issues etc.

[Martin-Molinero]

Just and update: Lean supports optimization since some time now https://github.com/QuantConnect/Lean/pull/4923 , genetic optimizer is not yet implemented though