search

bagaturchess / Bagatur

Java Chess Engine (UCI compatible)
http://bagaturchess.github.io/Bagatur/
Eclipse Public License 2.0
72 stars 18 forks source link

evaluation algorithm and mate #11

Closed tissatussa closed 2 years ago

tissatussa commented 3 years ago

hi, i just stumbled upon this position and i did some tests with several engines, including Bagatur :

bagatur_mate5_diagram

8/5q2/1p1K3p/7P/2k3P1/8/8/8 b - - 0 48

Black to move : Qf6+ is mate in 5. Many engines find this very fast, even some simple engines, often within ten seconds -SF 14 finds mate-in-5 within one second- but Bagatur seems struggling to find best move (i used MPV 7 and 1 in SCID) :

bagatur_mate_eval

bagatur_mate5_eval

bagaturchess commented 2 years ago

This is again interesting finding - thanks a lot!

I tested this position with many runs and it is reproducible on Dev version as well. In very rare cases Bagatur finds the mate in less than 10 seconds. In most cases it takes a long time - there was one run, where 700 seconds were necessary.

I found the main reason - the Late Move Reduction (LMR). It is quite "aggressive" in Bagatur. The formula gives 1 or even 2 moves reduction more than what I have seen in articles and other engines.

Such positions are very nice, because they are boundary positions, which well show if there is an heuristic in the engine, which is not adjusted well.

I could reduce the LMR reduction to find this mate in 5 moves but also fit in 10 seconds.

Is this ok from your perspective?

Best regards, Krasimir

tissatussa commented 2 years ago

In texts of chess engines i often read the term "LMR" .. i have only a vague understanding what this means, but i can imagine ..

What can be a good solution for Bagatur to solve this mate-in-# problem and other "boundary positions" ? I don't know, i'm not into engine programming .. if adjusting the LMR helps, it's OK with me .. but such change should not influence other functionality too much, i think. I leave it up to you. I can test the resulting new Bagatur version, and i can post other "irregular" position when i find them.

bagaturchess commented 2 years ago

Ok, thanks, I will decrease the LMR it in order to find this mate in up to 10 seconds each time.

The background: I find the reason relatively fast, because in the past I also found similar issues with LMR formula. I just prefer to see Bagatur going in bigger depths without decreasing its ELO rating and this is set to the possible maximum. My own explanation is that it perform bad in given positions but better in others and all these balances in regards to ELO.

But with this position and your tests it looks like only Bagatur cannot find the mate in up to 10 seconds - this has to be fixed.

P.S. This is a description of LMR, which looks short and well explained https://www.chessprogramming.org/Late_Move_Reductions

bagaturchess commented 2 years ago

I have just push my final changes for LMR. Everything looks fine from the last tournament with these changes:

image

tissatussa commented 2 years ago

honestly, i understand only 50% of these stats : is -7 the max ELO loss ? Not much then.

bagaturchess commented 2 years ago

Yes, we can fully ignore it, as it is below the standard error out of 95 games.

bagaturchess commented 2 years ago

I forgot to mention how this looks like from depth perspective: now Bagatur goes 3-5 moves shallower. I have to live with this :-)

image

bagaturchess commented 2 years ago

Hi, this needs more small adjustments and tests, because with more games the strength difference becomes -21 ELO. I have decrease it too much and now gave to shift it back a little bit. :-)

bagaturchess commented 2 years ago

This looks better, now it is only 1-2-3 moves shallower, doesn't have ELO decrease and solves the +5 mate position: image

bagaturchess commented 2 years ago

Happy New Year 2022! I wish good health and good mood! P.S. the version 3.0 is released and your feedback is welcome! ;-) I will close this issue.