search

official-stockfish / fishtest

The Stockfish testing framework
https://tests.stockfishchess.org/tests
282 stars 129 forks source link

Stochastic optimization for parameters #774

Open unaiic opened 4 years ago

unaiic commented 4 years ago

https://github.com/official-stockfish/Stockfish/issues/2915#issuecomment-678112885

Here some interesting methods were proposed. But regarding the first one, and after having mentioned to @nodchip, he told me about the implementation he used. I took his scripts and created a repo (https://github.com/unaiic/optimizer) where we can adapt them to SF and see how it goes. The scripts make use of Hyperopt, although we could also use Optune; we should see what is best in this case. I think you could help with this :)

joergoster commented 4 years ago

@kiudee Would it make sense to successively increase the number of games, or will this make all of the calculations so far useless?

kiudee commented 4 years ago

@joergoster I tried to answer the question here in the FAQ.

fsmosca commented 4 years ago

I have couple of updates on optuna tuner, now with skopt sampler from scikit with GP model along with different acquisition functions (acq_func) such as LCB, EI, PI and others. There are also explore and exploit factors on selected acq_func that can be interesting to play with.

This is the results so far at depth 6 up to 100 trials. Scroll below to see the summary.

Models such as GBRT (Gradient boosted regression trees) and ET (extra trees regressor) are not yet in the current repo will add it later.

vondele commented 4 years ago

@fsmosca feel free to test on fishtest optimized parameters, I've found that to be an essential step. In particular, I find that parameters might be improved at the TC (or depth) used in optimization, but fail at higher TC.

fsmosca commented 4 years ago

Thanks will do that once I get a promising param at longer TC. Currently exploring different optimization algorithm.

fsmosca commented 4 years ago

@joergoster

Another interesting way I found here https://facebookresearch.github.io/nevergrad/optimizers_ref.html#nevergrad.optimization.optimizerlib.ParametrizedOnePlusOne Allowing to re-evaluate the best point so far regularly, seems like a very interesting and promising way to me ...

I tried to implement that with a very interesting tuning. This is just a fixed depth 6 test to see how it behaves.

Command line

lakas.py --engine ./engines/stockfish-modern/stockfish.exe --budget 100 --concurrency 6 --games-per-budget 500 --input-param "{'RazorMargin': {'init':527, 'lower':250, 'upper':650}, 'FutMargin': {'init':227, 'lower':50, 'upper':350}}" --base-time-sec 30 --opening-file ./start_opening/ogpt_chess_startpos.epd --depth 6 --optimizer oneplusone

The param to be optimized

2020-10-06 18:11:57,175 | INFO  | input param: OrderedDict([('FutMargin', {'init': 227, 'lower': 50, 'upper': 350}), ('RazorMargin', {'init': 527, 'lower': 250, 'upper': 650})])

Setup

2020-10-06 18:11:57,175 | INFO  | total budget: 100
2020-10-06 18:11:57,175 | INFO  | games_per_budget: 500
2020-10-06 18:11:57,175 | INFO  | tuning match move control: base_time_sec: 30, inc_time_sec: 0.05, depth=6

Nevergrad optimizer

I set the optimistic as default for noise handling. ”optimistic”: the best optimistic point is reevaluated regularly, optimism in front of uncertainty

2020-10-06 18:11:57,175 | INFO | optimizer: oneplusone, noise_handling=optimistic, mutation=gaussian, crossover=False

Right on top it is already matching same params

2020-10-06 18:11:57,175 | INFO  | budget: 1
2020-10-06 18:11:57,175 | INFO  | recommended param: option.FutMargin=227 option.RazorMargin=527 
2020-10-06 18:11:57,175 | INFO  | default param: option.FutMargin=227 option.RazorMargin=527 
2020-10-06 18:12:31,868 | INFO  | actual result: 0.501 @500 games, minimized result: 0.499, pov: recommended_param

On later budgets

It is determined on targeting the param option.FutMargin=350 option.RazorMargin=250

2020-10-06 19:00:34,937 | INFO  | budget: 88
2020-10-06 19:00:34,937 | INFO  | recommended param: option.FutMargin=350 option.RazorMargin=250 
2020-10-06 19:00:34,937 | INFO  | default param: option.FutMargin=227 option.RazorMargin=527 
2020-10-06 19:01:08,227 | INFO  | actual result: 0.515 @500 games, minimized result: 0.485, pov: recommended_param

2020-10-06 19:01:08,242 | INFO  | budget: 89
2020-10-06 19:01:08,242 | INFO  | recommended param: option.FutMargin=350 option.RazorMargin=250 
2020-10-06 19:01:08,242 | INFO  | default param: option.FutMargin=227 option.RazorMargin=527 
2020-10-06 19:01:41,470 | INFO  | actual result: 0.503 @500 games, minimized result: 0.497, pov: recommended_param

2020-10-06 19:01:41,470 | INFO  | budget: 90
2020-10-06 19:01:41,470 | INFO  | recommended param: option.FutMargin=350 option.RazorMargin=250 
2020-10-06 19:01:41,470 | INFO  | default param: option.FutMargin=227 option.RazorMargin=527 
2020-10-06 19:02:14,804 | INFO  | actual result: 0.487 @500 games, minimized result: 0.513, pov: recommended_param

2020-10-06 19:02:14,820 | INFO  | budget: 91
2020-10-06 19:02:14,820 | INFO  | recommended param: option.FutMargin=350 option.RazorMargin=250 
2020-10-06 19:02:14,820 | INFO  | default param: option.FutMargin=227 option.RazorMargin=527 
2020-10-06 19:02:48,579 | INFO  | actual result: 0.501 @500 games, minimized result: 0.499, pov: recommended_param

2020-10-06 19:02:48,579 | INFO  | budget: 92
2020-10-06 19:02:48,579 | INFO  | recommended param: option.FutMargin=350 option.RazorMargin=250 
2020-10-06 19:02:48,579 | INFO  | default param: option.FutMargin=227 option.RazorMargin=527 
2020-10-06 19:03:22,992 | INFO  | actual result: 0.496 @500 games, minimized result: 0.504, pov: recommended_param

2020-10-06 19:03:22,992 | INFO  | budget: 93
2020-10-06 19:03:22,992 | INFO  | recommended param: option.FutMargin=350 option.RazorMargin=250 
2020-10-06 19:03:23,008 | INFO  | default param: option.FutMargin=227 option.RazorMargin=527 
2020-10-06 19:03:57,556 | INFO  | actual result: 0.509 @500 games, minimized result: 0.491, pov: recommended_param

2020-10-06 19:03:57,556 | INFO  | budget: 94
2020-10-06 19:03:57,556 | INFO  | recommended param: option.FutMargin=350 option.RazorMargin=250 
2020-10-06 19:03:57,556 | INFO  | default param: option.FutMargin=227 option.RazorMargin=527 
2020-10-06 19:04:30,800 | INFO  | actual result: 0.514 @500 games, minimized result: 0.486, pov: recommended_param

2020-10-06 19:04:30,808 | INFO  | budget: 95
2020-10-06 19:04:30,808 | INFO  | recommended param: option.FutMargin=350 option.RazorMargin=250 
2020-10-06 19:04:30,809 | INFO  | default param: option.FutMargin=227 option.RazorMargin=527 
2020-10-06 19:05:05,036 | INFO  | actual result: 0.502 @500 games, minimized result: 0.498, pov: recommended_param

2020-10-06 19:05:05,051 | INFO  | budget: 96
2020-10-06 19:05:05,051 | INFO  | recommended param: option.FutMargin=350 option.RazorMargin=250 
2020-10-06 19:05:05,051 | INFO  | default param: option.FutMargin=227 option.RazorMargin=527 
2020-10-06 19:05:38,169 | INFO  | actual result: 0.498 @500 games, minimized result: 0.502, pov: recommended_param

2020-10-06 19:05:38,184 | INFO  | budget: 97
2020-10-06 19:05:38,184 | INFO  | recommended param: option.FutMargin=350 option.RazorMargin=250 
2020-10-06 19:05:38,184 | INFO  | default param: option.FutMargin=227 option.RazorMargin=527 
2020-10-06 19:06:12,279 | INFO  | actual result: 0.513 @500 games, minimized result: 0.487, pov: recommended_param

2020-10-06 19:06:12,295 | INFO  | budget: 98
2020-10-06 19:06:12,295 | INFO  | recommended param: option.FutMargin=350 option.RazorMargin=250 
2020-10-06 19:06:12,295 | INFO  | default param: option.FutMargin=227 option.RazorMargin=527 
2020-10-06 19:06:46,631 | INFO  | actual result: 0.545 @500 games, minimized result: 0.45499999999999996, pov: recommended_param

2020-10-06 19:06:46,638 | INFO  | budget: 99
2020-10-06 19:06:46,638 | INFO  | recommended param: option.FutMargin=350 option.RazorMargin=250 
2020-10-06 19:06:46,639 | INFO  | default param: option.FutMargin=227 option.RazorMargin=527 
2020-10-06 19:07:21,183 | INFO  | actual result: 0.505 @500 games, minimized result: 0.495, pov: recommended_param

2020-10-06 19:07:21,199 | INFO  | budget: 100
2020-10-06 19:07:21,199 | INFO  | recommended param: option.FutMargin=350 option.RazorMargin=250 
2020-10-06 19:07:21,199 | INFO  | default param: option.FutMargin=227 option.RazorMargin=527 
2020-10-06 19:07:55,097 | INFO  | actual result: 0.516 @500 games, minimized result: 0.484, pov: recommended_param

The best after 100 budget

2020-10-06 19:07:55,113 | INFO | best_param: {'FutMargin': 350.0, 'RazorMargin': 250.0}

Game test

The tuning was done at depth 6, so the game test is also done at depth 6.

NeverGrad's param won.

Score of sf_ng_d6 vs sf_default: 4412 - 4097 - 1491  [0.516] 10000
...      sf_ng_d6 playing White: 2236 - 1999 - 765  [0.524] 5000
...      sf_ng_d6 playing Black: 2176 - 2098 - 726  [0.508] 5000
...      White vs Black: 4334 - 4175 - 1491  [0.508] 10000
Elo difference: 10.9 +/- 6.3, LOS: 100.0 %, DrawRatio: 14.9 %
Finished match
joergoster commented 4 years ago

@fsmosca Very interesting!

fsmosca commented 4 years ago

Lakas repo in github.

joergoster commented 4 years ago

@fsmosca Awesome! So much possibilities to try now ... :-)

ppigazzini commented 4 years ago

@fsmosca I saw that Lakas requires python 3.8.x, you could add a link to pyenv, see here an example to setup the latest python https://github.com/glinscott/fishtest/pull/778#issuecomment-702148654

fsmosca commented 4 years ago

@ppigazzini Thanks for the info on pyenv. I don't know about it. But will take a look at it.

I use pycharm as IDE. I also developed on windows 10.

ppigazzini commented 4 years ago

@fsmosca on windows is easy to install several python versions, on linux it's suggested to use pyenv and virtual environment to keep clean the system python.