search

kiudee / chess-tuning-tools

A collection of scripts aimed at efficiently tuning chess engine parameters.
https://chess-tuning-tools.readthedocs.io/en/latest/
Other
52 stars 13 forks source link

Variance Reduction does not sample from the ends of the intervals. #159

Closed Claes1981 closed 3 years ago

Claes1981 commented 3 years ago

Description

Describe what you were trying to get done.

Run Tuner with Variance Reduction acquisition function to explore the whole landscape and thereby also include as large range of scores/y-values as possible.

Tell us what happened, what went wrong, and what you expected to happen.

_The tuner never, 26 iterations so far, samples from the ends of the intervals, except for the first 5 n_initial_points iterations, even if no or almost no samples has been evaluated there earlier. At least with Variance Reduction I expect it with high probability to sample from those unexplored areas._

Plots of samples, parameter values on x-axes. The x-axes covers the whole ranges as specified in the json config. (You can find the exact values in the log, at "Got the following tuning settings:".) Iterations 16-25 was only with Variance Reduction acquisition function, and iterations 5-15 was a mix of Predictive Variance Reduction Search and Variance Reduction. (Scores on y-axes versus parameter values should be taken with a large grain of salt, since the other 3 parameter values are not shown in each plot respectively.) The table shows the values of iteration 0 to 25 in order.: Sample_ranges

10 rounds/20 games per iteration. no opening book. (I am still thinking of how to best implement a book only for engine2.) One out of four different opponents chosen at random each round.

Data: Cfish-20100303_pgo_extra_6t_h2048_4opponents_10rounds.npz.zip

Log: Cfish-20100303_pgo_extra_6t_h2048_4opponents_10rounds.log

Here is a plot at iteration 26 with no re-initialization (only fast-resume) since iteration 12: 20210816-055654-26

kiudee commented 3 years ago

It is indeed quite a counter-intuitive property of variance reduction (VR) to favor points in the center. This is due to it trying to pick points which reduce the predictive variance of all of the other points.

If you want to have a criterion which picks the single point with the highest uncertainty, then use "lcb" in conjunction with: https://github.com/kiudee/chess-tuning-tools/blob/2f489241e029b0d07ca6b645448643b4c57b4b83/tune/local.py#L394 alpha set to "inf".

I am planning to make these parameters configurable.

Claes1981 commented 3 years ago

Ah, thank you very much for the explanation. Yes, it would be interesting to see what "lcb" could do.

kiudee commented 3 years ago

Ah, thank you very much for the explanation. Yes, it would be interesting to see what "lcb" could do.

I only recently started to use lcb a lot, since it is able to correct a bad model fit. I am thinking about implementing something like acquisition function mixtures like

"acq_function": "{0.9: pvrs(n_thompson=1000), 0.05: lcb(alpha='inf'), 0.05: mean}",

where it would randomly choose between several acquisition functions according to the weights.

Claes1981 commented 3 years ago

Yes, something similar to the upstream gp_hedge. I have had about the same thought, but have not had the time to try to implement it yet. I was to begin with, only thinking about try to code some simple random choice between pvrs, mes, and ts, though. Your template seems to offer a lot more advanced fine tuning of the acquisition.

Until then, it seems there is no check currently in the Chess Tuning Tools code against passing "lcb" as an acquisition function? Since "lcb" should be supported by Bayes-skopt, I guess I can just try to pass "lcb" to the --acq-function option? And change alpha directly in the code.

kiudee commented 3 years ago

Yes, you can pass in "lcb", but to get "pure exploration" you also need to set alpha="inf" which is currently hardcoded to 1.96.

Claes1981 commented 3 years ago

Thanks, I will try to set alpha to "inf" on the line you showed above, while exploring the landscape with "lcb".

Claes1981 commented 3 years ago

I created a command line option to set the lcb alpha parameter: https://github.com/Claes1981/chess-tuning-tools/commit/1e6cc8a573da93085d0bd11be1984a0c05066c6a

(Or more accurateĺy: I mostly copy-pasted your code with acquisition_function_samples.)

Claes1981 commented 3 years ago

Actually I have a problem passing inf OR "inf" on the command line. In both cases it seems this line does not become true. (I placed a breakpoint below the line which never triggered, but printing alpha on the line above outputs inf in both cases.) I wonder if it has something to do with alpha being a float but might be compared to a string on the line...

Update: Worked around it now by commit https://github.com/Claes1981/chess-tuning-tools/commit/0dad57e9f42e22fd208419e78d57840af65ba579, it seems to be working now.

Claes1981 commented 3 years ago

Yes, something similar to the upstream gp_hedge. I have had about the same thought, but have not had the time to try to implement it yet. I was to begin with, only thinking about try to code some simple random choice between pvrs, mes, and ts, though. Your template seems to offer a lot more advanced fine tuning of the acquisition.

I made a simple random selection of acquisition function each iteration possible: https://github.com/Claes1981/chess-tuning-tools/commit/59b92587bc87fe7376e5349cab16724642f595b5 Hoping it somehow can combine all good parts of the different acquisition functions at once. (Possibly it also combines the bad parts of each acquisition function...)