search

dme65 / pySOT

Surrogate Optimization Toolbox for Python
Other
205 stars 53 forks source link

stopping criteria and failed evaluation question #17

Closed rho80 closed 7 years ago

rho80 commented 7 years ago

Hi there, to be sure, currently the only one stopping criteria supported in pySOT is maximum number of evaluations, am I right? The second question is about failed evaluation. What should my own "objfunction" method return to consider it as failed evaluation by pySOT? Should "ValueError" exception arise? And what about optimization process behavior in this case, does it stop or continue? (I've read the issues #8 and #9)

dme65 commented 7 years ago

Hi! 1) That is correct. Are you interested in using another stopping criterion? I would be happy the add some and doing this shouldn't require much work on my part. 2) This example should be helpful: https://github.com/dme65/pySOT/blob/master/pySOT/test/test_subprocess.py pySOT will try to go back and evaluate this input again if something went wrong the first time.

rho80 commented 7 years ago

Thanks for information and for the example. Regarding to stopping criterion: especially during testing stage (which means before the final start of (possibly) very time consuming optimization process) I'd welcome "real time" criterion - something like run till Monday morning ;). And sometimes it'd be useful to have criterion related to fitness value (absolute value or no improvement after specified number of evaluations for example).

SanPen commented 7 years ago

Hi, +1 to a stopping criteria.

Outstanding work by the way.

dme65 commented 7 years ago

@dbindel: Is there support for this in POAP?

@SanPen: I'm glad you like pySOT!

dme65 commented 7 years ago

A real-time stopping criterion will be added to the next version.