Termination criteria does not seem to work

[CharlyEmpereurmot]

Hello Marco,

I have an issue with the termination criteria. Basically, setting them has no effect and my code continues optimizing forever.

So I do this first, which is OK:

# eval_function works fine and boundaries are OK, I'm sure of this
FP = FuzzyPSO()
FP.set_search_space(search_space_boundaries)
FP.set_fitness(fitness=eval_function, arguments=None, skip_test=True)

And then I do any of these:

result =  FP.solve_with_fstpso()
result =  FP.solve_with_fstpso(max_iter=1, max_iter_without_new_global_best=1)
result =  FP.solve_with_fstpso(max_iter=100, max_iter_without_new_global_best=100)

And it continues optimizing forever. Looking at your code I really do not understand why it's not working.

Do you have any ideas?

I really appreciate your help, I'm almost done with this project now and the implementation of the initial guesses was key to making the whole thing work.

[aresio]

Hi Charly, thank you for the information, I will investigate this issue immediately

[CharlyEmpereurmot]

Are you able to reproduce this behavior that never terminates? I say this because there is a probability I triggered this behavior, but I believe this is not the case.

[aresio]

The argument "max_iter_without_new_global_best" is not actually supported by FST-PSO. I am implementing it right now, because it was among the features that I wanted to add in the next future.

[CharlyEmpereurmot]

Actually, even these do not terminate for me:

result =  FP.solve_with_fstpso(max_iter=1)
result =  FP.solve_with_fstpso(max_iter=100)

[aresio]

That is actually unexpected, the max_iter argument is supposed to work. Let's hope the new version fixes the issue!

I just uploaded the new version (1.7) on pypi. Can you please try it out?

[CharlyEmpereurmot]

I'm now using 1.7.1 and it still does not terminate for my custom code.

However, I retried your example code which terminates fine whether I use max_iter or max_iter_without_new_global_best. So thank you for implementing/fixing max_iter_without_new_global_best, it works !

I still have to figure why my code is not stopping correctly, I will investigate more but nothing stands out atm .. super strange.

[aresio]

I will perform some additional termination tests in the afternoon. It might be related to the new budget allocation routines.

[CharlyEmpereurmot]

So I ran more tests and it actually ~works, but probably there is just a small error and/or I'm misinterpreting some things. Using parameters:

3 particles (forced)
max_iter = 1
max_iter_without_new_global_best = 1

It performs a total of 9 evaluations of the fitness function. My understanding was that with these settings FST-PSO would perform 3 evaluations of the fitness function (possibly 6 if you consider that you first initialize the swarm), but 9 seemed wrong to me.

I was running long fitness evaluation with > 20 particles, so I did not understand at first.

Below is some simplified output of the verbose mode:

* Max distance: 66332.495808
 * Search space boundaries set to: [[0.225, 0.27], [0.18, 0.215], etc]
 * Max velocities set to: [0.045, 0.035, etc]
 * Number of particles automatically set to 19
 * Swarm size now set to 3 particles (forced 3 particles)
 * Maximum number of iterations without any update of the global best set to 1
 * Maximum number of iterations set to 1
 * Enabled settings: [cognitive] [social] [inertia] [minvelocity] [maxvelocity]

 *** Launching optimization ***
 * Creating and evaluating particles
 * 3 particles created.
 * FST-PSO will now assess the local and global best particles.

FITNESS EVAL NB: 1 # print from my eval function
FITNESS EVAL NB: 2
FITNESS EVAL NB: 3

 * New best particle in the swarm is #0 with fitness 240.998000 (it: 0).
Process started
 * Iteration: 0
   since last global update: 0

FITNESS EVAL NB: 4
FITNESS EVAL NB: 5
FITNESS EVAL NB: 6

Starting verification of local best
 Solution 0 : 0.2571345856438336    0.18813189144874304 etc
 Fitness calcolata: 247.063 old best 240.998
 Solution 1 : 0.240087211248572 0.18714218687880663 etc
 Fitness calcolata: 477.455 old best 474.767
 Solution 2 : 0.2352166861792658    0.18234145283191924 etc
 Fitness calcolata: 342.846 old best 330.641
Completed iteration 1
 * Iteration: 1
   since last global update: 1

FITNESS EVAL NB: 7
FITNESS EVAL NB: 8
FITNESS EVAL NB: 9

Starting verification of local best
 Solution 0 : 0.2571345856438336    0.18813189144874304 etc
 Fitness calcolata: 247.063 old best 240.998
 Solution 1 : 0.246837211248572 0.18878792607785588 etc
new best for  1  has fitness 365.247
 Solution 2 : 0.2419666861792658    0.18365465414980464 etc
new best for  2  has fitness 236.208
 * New best particle in the swarm is #2 with fitness 236.208000 (it: 1).
Completed iteration 2
 * Iteration: 2
   since last global update: 1
 * Maximum iterations reached.
Process terminated, best solution found: [0.2419666861792658, 0.18365465414980464, etc] with fitness 236.208
Best solution: 0.2419666861792658   0.18365465414980464 etc
Whose fitness is: 236.208

TOTAL EVALS: 9 # prints from my code
Total execution time: 0.06 h
Total fitness eval time: 0.06 h (100.0 %)

[CharlyEmpereurmot]

Otherwise I think the termination criteria work just fine

[aresio]

Okay, then the problem is that it is performing an additional wrong "iteration". That's a bug probably introduced by a new functionality, I'll release a patch in the next hours. Thanks!

[CharlyEmpereurmot]

Yes it is only this, nothing terrible. Truth is at some point I actually got confused between nb of swarm iterations and nb of evaluations, sorry!

[aresio]

In this new version you can set the number of evaluations too, but that can be tricky. I will prepare a specific documentation on that.

Meanwhile, I pushed version 1.7.3 that fixed the additional iteration problem.