Grid search variant - Githubissues

SimonBlanke / Gradient-Free-Optimizers

Simple and reliable optimization with local, global, population-based and sequential techniques in numerical discrete search spaces.

https://simonblanke.github.io/gradient-free-optimizers-documentation

MIT License

1.21k stars 84 forks source link

Grid search variant #10

Closed SimonBlanke closed 2 years ago

SimonBlanke commented 3 years ago

I think it would be useful to have a grid search optimizer in this package. But its implementation would probably be quite different from other ones (sklearn, ...).

The requirements are:

The grid search has to stop after n_iter instead of searching the entire search space
The positions should not be precalculated at the beginning of the optimization (i have concerns about memory load).

djokester commented 3 years ago

@SimonBlanke let me start with this.

SimonBlanke commented 3 years ago

@djokester I would like to give you some guidance for this task:

You can start by creating the class "GridSearch" and inherit both the BaseOptimizer and Search. You can use random search as a template for this.

You should also keep in mind that each optimizer has the following methods:

init_pos (initializes the positions given in "initialize". In most cases this can just be inherited from BaseOptimizer)
finish_initialization (most optimizers don't use this. Just inherit this from BaseOptimizer)
iterate (very important! This returns the next position)
evaluate (what to do after the score is calculated for the new position. E.g. move to the new position if better than current.)

For the GridSearch init_pos, finish_initialization and evaluate can be inherited from BaseOptimizer. "evaluate" doesn't need to be implemented for GridSearch because it does not decide what to do next based in the new score.

This is all I figured out about the implementation. I hope this helps!

djokester commented 3 years ago

Yeah @SimonBlanke. Thanks it helps a lot

SimonBlanke commented 3 years ago

Hello @djokester,

have you made some progress on the grid search algorithm? If you need some additional assistance I would be happy to help.

djokester commented 3 years ago

@SimonBlanke I have gone through the code base. I will send a PR by this weekend.

SimonBlanke commented 3 years ago

@djokester a PR would be nice! You can ignore the extended tests (performance, parameters, ...), I will take care of those. If you have a basic concept we can continue from that together.

cryptocoinserver commented 2 years ago

@djokester Any news? Thank you for your time!

tgdn commented 2 years ago

@cryptocoinserver we have an open PR #18, let us know what you think!

SimonBlanke commented 2 years ago

@tgdn and his team @EdouardVilain-Git, @tabeare, @annabieber, @MachineLearner75 provided an implementation of grid-search, which can be used in Gradient-Free-Optimizers >= v0.5.0.

You can read more about it in the optimization-tutorial.