godatadriven / evol

a python grammar for evolutionary algorithms and heuristics
https://evol.rtfd.io
MIT License
187 stars 12 forks source link

adding custom stop conditions to the evol pipeline #156

Closed GiliardGodoi closed 3 years ago

GiliardGodoi commented 4 years ago

Some classes define stop conditions for the evolutionary process such as MinimumProgress or TimeLimit. Perhaps there is a gap in how these conditions can be added to the pipeline, though.

A condition instance is added to conditions set (a Condition class property) in the dunder methods __enter__ and __exit__, which define a context manager. The class method 'check' verify each condition in the conditions set for a given population. This method (check) is executed at the 'BasedPopulation' method evolve.

To the best of my knowledge, context managers work with the "with ... as" statement. So the Condition object should be used something like:

with TimeLimit(1000) as condition:
    # do something
    ...

However, the current evolve method just executes the pipeline for a predefined number of iterations. The StopEvolution exception might never be raised because there isn't a straight way to add other conditions.

Let's pretend we want to test the efficiency of different operators - crossover, mutation, or selection operators. To compare their efficiency some metrics can be used such as the number of times a specific stop condition was reached (like ReachedBestKnowSolution, MinimumProgress, and IterationLimit).

Thus, we have two questions here:

  1. How can we add several stop conditions to the evolutionary pipeline? (following the same principles that Evol lib was designed)
  2. And how to keep track of which stop condition was reached? But I think this question is more relatable to the "adding properly log" problem mentioned in the README.

BTW English is not my primary language so bear with me guys!

GiliardGodoi commented 4 years ago

Later, I've realized that in tests\test_conditions.py there are some example of how to use a Condition object, but to use multiples conditions we should do something like that:

with Condition(lambda pop: pop.generation < max_generation):
    with TimeLimit(seconds) :
        with ReachedBestKnowSolution(nro):
            result = pop.evolve(evo)

Or test multiples conditions in only one function.

def multiples_stop_conditions(pop):
    if pop.generation >= max_generation:
         return False
    elif pop.current_best.fitness ==best_known_solution :
         return False

    return True

with Condition(multiples_stop_conditions):
    result = pop.evolve(evo)

Isn't there a better way to express these conditions?

koaning commented 4 years ago

I think your latter solution is the one I like the most.

So technically you can also use a callback and catch the error what-ever way you like.

evo = (Evolution()
  .mutate(...)
  .surive(...)
  .breed(...)
  .callback(lambda p: ... raise your own error if it makes sense ...))

try:
     pop.repeat(evo, 100)
except TimeLimitException:
    # handle it however you like 
except GenerationLimit:
    # handle it however you like 

@rogiervandergeer got any comments?

GiliardGodoi commented 4 years ago

Hi guys! I saw at test code's files that we can add multiples stop conditions using this syntax:

 with Condition(lambda pop: False):
            assert len(Condition.conditions) == 1
            with TimeLimit(60):
                assert len(Condition.conditions) == 2

We can do something like this too:

with Condition(lambda pop: False), \
        TimeLimit(60), \
        MinimumProgress(window=10) : 

        pop.evolve(evo, n=5)

The with operator accept multiples object separated by a comma.

To catch which stop condition was reached, I just passed a message for the StopEvolutionException.

In the method evolve I do a slight change to something like:

        try:
            for _ in range(n):
                Condition.check(result)
                for step in evolution:
                    result = step.apply(result)
        except StopEvolution as reason:
            why_stopped = reason.value or str(reason)

In the condition class, I add a msg attribute, which is passed to the StopEvolutionException.

It turns out to be pretty easy to add multiples stop conditions to the evolutionary process.

GiliardGodoi commented 3 years ago

I think this question was answered, and some changes I've made in my own fork.