Migrating from pgpelib to evotorch

[apolinario]

Hi! I am a "mindless" user of pgpelib (using it without really understanding what is down the hood). I saw the release of evotorch and I am wondering if it is worth swapping in all cases. And I would like to also understand how could I adapt an existing pgpelib code to evotorch if that even makes sense

Currently the code I have does something like:

pgpe = PGPE(
    solution_length= SOLUTION_LENGTH,   # A solution vector has the length of 5
    popsize= NUM_SOLUTIONS,          # Our population size

    optimizer='clipup',          # Uncomment these lines if you
    optimizer_config = dict(     # would like to use the ClipUp
       max_speed=0.15,           # optimizer.
       momentum=0.9
    ),
)

And then to run I do it like this

for generation in range(1000):
    # create new solutions
    solutions = pgpe.ask()

    # evaluate fitnesses
    fitnesses = [evaluate_solution(genes) for genes in solutions]

    # update the population  
    pgpe.tell(fitnesses)

    # Saving the best in the generation
    fitness, canvas = evaluate_solution(pgpe.center, returns_image=True)
    canvas = draw_fitness(canvas, generation, fitness)

    # Store the best 
    if fitness > best_fitness:
      best_fitness = fitness
      best_generation = generation
      best_solution = pgpe.center
fitness, canvas = evaluate_solution(best_solution, returns_image=True)
canvas = draw_fitness(canvas, generation, fitness)

It wasn't super clear how best to adapt it to evotorch or even if it would be beneficial to do so. Thank you!

[engintoklu]

Hello @apolinario, and thank you for your interest in EvoTorch!

We recommend switching to EvoTorch. In general, you might want to consider switching for these reasons:

• EvoTorch, as its name implies, is based on PyTorch. Thanks to this, it can take advantage of GPU-accelerated computation.
• EvoTorch was written with vectorized computation in mind. Even when not using GPU-acceleration, significant performance boost can be observed with EvoTorch compared to pgpelib in a lot of scenarios.
• EvoTorch is also powered by the ray library and has built-in distributed computation capabilities. With the help of these capabilities, evaluation of large populations can be split into multiple CPUs or multiple GPUs.
• EvoTorch supports multiple evolutionary algorithms (not just PGPE).
• EvoTorch supports multi-objective optimization.

You might want to see the documentation and the examples where various features of EvoTorch are demonstrated.

I think your example code could look like this after switching to EvoTorch:

from evotorch import Problem
from evotorch.algorithms import PGPE
from evotorch.decorators import vectorized
from evotorch.logging import StdOutLogger
import torch

@vectorized
def fitness_function(solutions: torch.Tensor) -> torch.Tensor:
    # At this point, `solutions` is expected as a PyTorch tensor of shape
    # (N, L), where N is the number of solutions, and L is the length of a
    # solution.

    # For the sake of this example, let us define the fitness function
    # as the norm of a solution.
    return torch.linalg.norm(solutions, dim=-1)

SOLUTION_LENGTH = 10
NUM_SOLUTIONS = 100

problem = Problem(
    "min",               # The goal is to minimize
    fitness_function,    # This is the fitness function
    initial_bounds=(-10.0, 10.0),  # Sample new solutions from within -10...10
    solution_length=SOLUTION_LENGTH,
    dtype=torch.float32,  # Express the solutions using the dtype torch.float32
)

pgpe = PGPE(
    problem,
    popsize=NUM_SOLUTIONS,

    # The following are example hyperparameter values.
    # It is recommended to tune them according to the problem at hand.
    center_learning_rate=0.075,
    stdev_learning_rate=0.1,
    stdev_init=0.1,
    stdev_max_change=0.2,
    optimizer="clipup",
    optimizer_config = dict(
        max_speed=0.15,
        momentum=0.9,
    ),
)

canvas = ...

def draw_fitness(canvas, generation, fitnesses):
    ...

# Print status to the standard output at every 10 generations
_ = StdOutLogger(pgpe, interval=10)

for generation in range(1000):
    pgpe.step()
    fitnesses = pgpe.population.evals[:, 0]
    canvas = draw_fitness(canvas, generation, fitnesses)

print("Final status:", dict(pgpe.status))

I hope this helps! Feel free to let me know if something is not clear.

Happy coding!

[Higgcz]

Closing due to inactivity, feel free to reopen if you have more questions.