Performance boost using Numba

[TiemenSch]

I've been using DEAP for my project for a while now and I was investigating performance using Visual Studio's Python profiler.

First, I tweaked my own code's performance using Numba and since I'm doing a lot of list/matrix indexing it made it run tons and tons faster.

Numba gives you the power to speed up your applications with high performance functions written directly in Python. With a few annotations, array-oriented and math-heavy Python code can be just-in-time compiled to native machine instructions, similar in performance to C, C++ and Fortran, without having to switch languages or Python interpreters.

Numba works by generating optimized machine code using the LLVM compiler infrastructure at import time, runtime, or statically (using the included pycc tool). Numba supports compilation of Python to run on either CPU or GPU hardware, and is designed to integrate with the Python scientific software stack.

Then I investigated the different functions in DEAP and actually a lot of them are eligible for Numba's default @jit decorator. This decorator automatically compiles the function (if eligible) just-in-time.

That "if eligible" is important, as it does not work out of the box for certain operations. If it's not eligible, it's easy to find the function it failed on from the error stack.

In my custom library, I've changed the following to work with @jit:

• individual generation (each individual is a list of 4 elements, uniformly sampled)
• evaluation involving clustering algorithms and modularity indices (my actual project)
• mating (crossover)
• mutating

So the ones currently not using @jit roughly are toolbox generation, stats, halloffame and the call to eaSimple.

Of course, this would add a new dependency to the project. That could be circumvented using an optional import, though.

Let me know what you think!

[evolvingfridge]

Hey, I never used numba myself yet, are you using @jit for every method in a class ?

[TiemenSch]

Here are my custom mate and mutate functions as an example:

import random
from typing import List
from numba import jit

ind_type = List[float]

@jit
def mate_params(ind1: ind_type, ind2: ind_type,
                max_ratio: float=0.5) -> (ind_type, ind_type):
    """Mate two parameter lists to a sampled ratio.

    Arguments:
        ind1, ind2: Individuals to mate.
        max_ratio: Maximum ratio (uniform sample from 0.0).

    Returns:
        Blended individuals.
    """
    rat = random.uniform(0.0, max_ratio)
    rem = 1 - rat

    for i, (v1, v2) in enumerate(zip(ind1, ind2)):
        ind1[i] = rem * v1 + rat * v2
        ind2[i] = rat * v1 + rem * v2

    return ind1, ind2

@jit
def mutate_params(indpb: float, sigma: ind_type,
                  lbs: ind_type, ubs: ind_type, ind: ind_type) -> (ind_type,):
    """Gaussian mutation on a parameter set.

    Arguments:
        indpb: Independent probability for each attribute to be mutated.
        sigma: Standard deviations for the gaussian addition mutation.
        lbs: Lower bounds on entries in individual.
        ubs: Upper bounds on entries in individual.
        ind: Individual to mutate.

    Returns:
        A tuple of one individual.
    """

    for i, s in enumerate(sigma):
        if random.random() < indpb:
            ind[i] += random.gauss(0.0, s)
            if ind[i] < lbs[i]:
                ind[i] = lbs[i]
            elif ind[i] > ubs[i]:
                ind[i] = ubs[i]

    return ind,

[evolvingfridge]

thanks for posting code, I should try jit one day :)

[TiemenSch]

I'm currently finishing my master's thesis, but could try to fork a numba branch some day in March/April and check the performance increase on my system.

Is there an example you would like to see/use as a benchmark case?

[robertrenecker]

This isn't using nopython mode correct? I'm trying to use nopython mode for decorators but having trouble with dealing with <class 'deap.creator.Individual'> types