Closed atrettin closed 2 years ago
I forgot to mention: Algorithm-specific parameters (https://nlopt.readthedocs.io/en/latest/NLopt_Reference/#algorithm-specific-parameters) are passed as a dictionary to the keyword algorithm_params
.
Could the above useful description be added to the code as docstring? Then this will be handy and also render nicely in the online pisa docs. Other than that this PR looks good to me 👍
I added a lot of documentation and formatted the docstrings in such a way that they will actually be rendered nicely when building the docs as HTML.
I am happy to merge this, it adds nlopt
support and a lot of documentation, while not changing any of the existing code
Support for all gradient-free global and local optimization routines from the NLOPT package
With this PR I add a new option to the box of algorithms available in PISA that allows the user to use any of the global (https://nlopt.readthedocs.io/en/latest/NLopt_Algorithms/#global-optimization) and local (https://nlopt.readthedocs.io/en/latest/NLopt_Algorithms/#local-derivative-free-optimization) algorithms. The reason why I only chose to support the gradient free methods is that NLOPT doesn't apply a finite-difference method automatically and expects every function to supply its gradient itself. Most gradient based algorithms are also included in scipy anyway and for our purposes the gradient-free methods and global searches are more interesting.
NLOPT can be dropped in place of scipy and iminuit by writing a dictionary with
"method": "nlopt"
and choosing the algorithm by its name of the formNLOPT_{G,L}{N}_XXXX
. For example, to use the Nelder-Mead algorithm, you would writeand then run the fit with
. Of course, you can also nest the
nlopt_settings
dictionary in any of thefit_octants
,fit_ranges
and so on by passing it aslocal_fit_kwargs
.Adding constraints
Adding inequality constraints to algorithms that support it is possible by writing a lambda function in a string that expects to get the current parameters as a
ParamSet
and returns a float. The result will satisfy that the passed function stays negative (to be consistent with scipy). The string will be passed toeval
to build the callable function. For example, a silly way to bounddelta_index
> 0.1 would be:Adding inequality constraints to algorithms that don't support it can be done by either nesting the local fit in the
constrained_fit
method or to use NLOPT's AUGLAG method that adds a penalty for constraint violations internally. For example, we could do this to fulfill the same constraint with the PRAXIS algorithm:Using global searches with local subsidiary minimizers
Some global searches, like evolutionary strategies, use local subsidiary minimizers. These can be defined just as above by passing a dictionary with the settings to the
local_optimizer
keyword. Note that, again, only gradient-free methods are supported. Here is an example for the "Multi-Level single linkage" (MLSL) algorithm, using PRAXIS as the local optimizer:For some evolutionary strategies such as ISRES, the
population
option can also be set.