search

SimonBlanke / Gradient-Free-Optimizers

Simple and reliable optimization with local, global, population-based and sequential techniques in numerical discrete search spaces.
https://simonblanke.github.io/gradient-free-optimizers-documentation
MIT License
1.21k stars 84 forks source link

DIRECT optimizer #8

Closed SimonBlanke closed 2 years ago

SimonBlanke commented 3 years ago

Multiple users requested DIRECT optimizer in this package. This issue will track the process of its implementation and improvement.

djokester commented 3 years ago

Hey @SimonBlanke! Do you mind if I work on this? if you have started working on this maybe you can give me something else to work on?

SimonBlanke commented 3 years ago

Hello @djokester,

you are very welcome to work on this (or another) issue. I have not started working on this algorithm yet. But there are some things you should keep in mind:

Implementing new optimization algorithms is usually quite difficult for other developers because there are some strict requirements to make them work in the same API. Because of this all optimizers have something like a "backend API". All optimizers have the following methods:

Those are then used in search.py. Also: The code is currently not well documented and explained.

If you want another task that would be a better introduction to this package I opened multiple new issues.

SimonBlanke commented 2 years ago

Hello @djokester,

I was able to implement the direct search, based on the explanation in this fantastic article. It was much easier to understand this algorithm after I understood and implemented the Lipschitz Optimizer, but it was quite the challenge!

It will be released in v1.2 and can already be seen in the corresponding branch. If you like you can take a look at the source-code and give suggestions.

This is the search-path in a convex optimization-problem:

DirectAlgorithm

SimonBlanke commented 2 years ago

this feature was released in v1.2