[FEATURE] save&load (e.g., pickle) population snapshot easily based on certain criteria

[oni73]

Hi,

i couldnt find any example to be able to save&load the population during evolving.

For example, something like this would be really useful:

pseudo code:

for i in range(n_iterations):
    pop =  evolve(pop)
    if pop.champion_f < best_f:
         save_snapshot(pop, i)

I wonder how / if this wold be easily possible with pygmo. Any example would be highly appreciated!

P.S: Would this way of evolving break any adaptive DE variants ? Many Thanks

[bluescarni]

@oni73 apologies for the delay in the reply.

As long as you are using directly the algorithm API (as you show in your example), manually pickling after every evolve based on whatever criterion you like should "just work".

Are you asking about a higher-level API? I am not sure about the added value of a higher-level facility with respect to your code snippet, but I am open to suggestions.

[oni73]

Hi @bluescarni ,

many thanks for your reply. Indeed, i can confirm that the proposed approach works and this way we have quite fine-grained control over the evolution process....

what i ended up using is the following:

prob=pygmo.problem()
   n_iterations=100
   popsize=40
   algo = pygmo.algorithm(pygmo.de1220(gen=1))
   pop = pygmo.population(prob, popsize)
   for i in range(n_iterations):
       pop = algo.evolve(pop)
       if pop.champion_f < best_f:
           best_f = pop.champion_f
           save_population(pop, 'experiment-1', i, best_f)

where save_population and load_population are defined as:

    def save_population(pop, exp_name, iteration: int, bestf: float):
        pop_snapshot_fname = exp_name + '/' + 'population-' + str(iteration) + '-' + str(bestf)
        np.savez_compressed(pop_snapshot_fname, x=pop.get_x(), fitnesses=pop.get_f())

    def load_population(pop_snapshot_path: str, prob, popsize: int):
        print('loading population snapshot', pop_snapshot_path, '...')
        xfs = np.load(pop_snapshot_path)

        pop = pygmo.population(prob=prob)

        for i in range(popsize):
            pop.push_back(x=xfs['x'][i], f=xfs['fitnesses'][i])
        print('done.')

        return pop

IMO, it would be beneficial to provide library support for such functionality.

[EveCharbie]

Hi :) Thank you @oni73 for sharing the code. I have a large problem to run (weeks or months of computation). I am concern that if something happens and the code gets interrupted (ex we loose power), I will loose all the progress made. From the code above, I see that each evolution of the population can be saved. So I can load the results form the last population evolution before interruption. My question (@bluescarni) is : Is it possible to warm start a problem? I would provide the starting point and the optimization would resume from there.