WandererXII / lishogi

☗ lishogi.org: the forever free, adless and open source shogi server forked from lichess.org ☗
https://lishogi.org
GNU Affero General Public License v3.0
263 stars 40 forks source link

Engine's Judgement #789

Open HansAldiC opened 6 months ago

HansAldiC commented 6 months ago

https://lishogi.org/NJ7SxP7f#48 I was wondering why the engine finds the move I made (move 49) as an inaccuracy eventhough the evaluation bar stooped quite significant (-2.5 to -7.4). I would expect this should be regarded as a mistake.

Screenshot_20240228_004610_Chrome

p.s I was wondering when will the next update be announced. Would really appreciate if you could share anything. Thanks.

WandererXII commented 6 months ago

Thanks for reporting this. It's possible this might need some tweaking, but currently it's calculated like this:

We take the previous (-2.5) and the current CP evaluation (-7.4) and transform them to winning chances, using this:

def cpWinningChances(cp: Double): Double = 2 / (1 + Math.exp(-0.0007 * cp)) - 1 

(https://github.com/WandererXII/lishogi/blob/master/modules/analyse/src/main/Advice.scala#L63) CP is not used directly because losing 1000 CP when the game is equal is much worse then losing 1000 CP when you are winning with +5000CP for example.

Winning chance is a scale from -1 to 1. Here it gives us cca -0.1 and -0.25. We get the difference which is 0.15. Now if the difference is larger than 0.3 it's a blunder, if it's larger than 0.2 it's a mistake and if it's larger than 0.1 it's an inaccuracy. So here we get just an inaccuracy.

Now this 0.1, 0.2, 0.3 was taken from lichess, so perhaps different numbers could be used, but it's hard to get the right balance, we don't want to make it too sensitive or too forgiving.

p.s I was wondering when will the next update be announced. Would really appreciate if you could share anything. Thanks.

Hopefully soon, I'm currently finalizing and testing everything, I would still like to take few more days for testing, since I keep finding bugs, but I'm hopeful that I could have the new update up this week. But no promises for now.