search

lightvector / KataGo

GTP engine and self-play learning in Go
https://katagotraining.org/
Other
3.56k stars 564 forks source link

Horizon effect #601

Open michito744 opened 2 years ago

michito744 commented 2 years ago

A phase that has been covered in past issue.

It is very very cheep... 2022-02-18

With a little assistance, KataGo can reach the right conclusion, but... 2022-02-18 (1) The weighting of the policy is so biased that there is almost no chance of discovering it on your own in the first place.

HackYardo commented 2 years ago

How do you know it isn't a better choice to move K2 instead of F18? In my view, F18 is a trivial solution while K2 is a singular solution because White will have a huge authority around L7 if K2 is chosen.

michito744 commented 2 years ago

The bottom side is already okay with W doing nothing. If Black actually move K2, KataGo himself can discover the procedure.

michito744 commented 2 years ago

Details

Black's bottom side has already been crashed. 2022-02-21 Even if you start with black, W6 is a excellent move after B1W2B3W4B5, and either △ or □ key stones will be destroyed.

The horizon effect is caused by policies that fail to detect this W6 preferentially.

michito744 commented 2 years ago

Anomalies are sometimes the best solution in Go, but KataGo's policy is significantly lacking in the ability to discover it.

michito744 commented 2 years ago

@HackYardo In your phrase, White's singular solution is a “pass” in the bottom side.

HackYardo commented 2 years ago
I download the sgf from go4go and analyze it under Japanese rules 6.5 komi KataGo 20 block 1600 playouts(it is a superhuman Go player from one of the KataGo's paper's appendices), here is the conclusion: W B winrate of W scorelead of W
F18 K2 58 0.1
K2 F18 78 2.4

Clearly, K2 is better than F18 for W. If you say a AI's blind spot or the AI is wrong, you mean a better way to do something that makes the AI shocked.

Apart from AIs, in a human view, I think there is a "initiative fight", B hopes W to move F18, W says "No" and make trouble for L6,L5,Q7.

michito744 commented 2 years ago

Hahaha.

With about 1600 playouts in 20 blocks, it is far from a linear search by a human player of the highest level. KataGo is only superior to humans in terms of breadth of vision and judgment of the shape of the game in stable phases, and is only at the level of amateur players about detecting move in parts. (To get results of high-level professional player, you need to optimize the settings and search the game hundreds of thousands of times or more.)

In the first place, in the first diagram, it was obvious that the black bottom side was already crushed in the progression (Black's win rate was almost 5%), so Black avoided this change in the actual game. See the second diagram.

If you can't understand that black is crushed in the third diagram, further discussion is meaningless.

lightvector commented 2 years ago

Can you provide an SGF?

michito744 commented 2 years ago

The game record is pasted in the comment I posted on January 23 of Issue #456.

HackYardo commented 2 years ago

@michito744 I can understand your three positions in 30sec... It seems like a beginner's level, and it is far from the original game played by Ichiriki Ryo... __go4go_20220121_Ichiriki-Ryo_Iyama-Yuta.sgf.txt

Yes, AIs can not handle part board well, it is because AIs always focus on whole board. That's the point, if you analyze the position from a part view, you cannot understand it, you should analyze it from a whole view.

michito744 commented 2 years ago

The more accurately we play move the part, the higher the winning percentage will be. In some cases, the victory or defeat will be overturned.

The reason why Ikiriki avoided entering this change in B45 was because he knew that Black would surely lose if he entered the change discussed here.

On the other hand, neither KataGo nor FineArt could foresee this change correctly, and they were not afraid to offer the entrance to the crush course as the best move for an even game. To put it bluntly, it was fool.

michito744 commented 2 years ago

@HackYardo If W6 in third diagram is really beginner level, then KataGo is less than that.

Then, and this is a fundamental problem of AI, if the phase is even slightly different, AI needs to start searching for the parts all over again.

Humans can easily cope with difficult progressions because they save resources by locking and reusing the results of parts that have been explored once.

If we want AI to have the same level of responsiveness, it must be able to solve the correct solution of a part at a glance (that is, it must be overwhelmingly more accurate than humans in predicting parts).