search

axiomizer / haida-go

An implementation of a go (baduk) AI, following methods published for AlphaGo Zero. The primary purpose is to gain hands-on experience with machine learning.
0 stars 0 forks source link

L2 Regularization #1

Closed axiomizer closed 1 year ago

axiomizer commented 1 year ago

image

The "Methods -> Optimization" section mentions that c = 10^-4

Mastering the game of Go without human knowledge

axiomizer commented 1 year ago

Implemented in 80ec767266f050f3e39da453b14033ba978b6c5a.

In the unit tests, I'm passing in weight_decay to torch.optim.SGD, but even if nothing is passed in, the test won't fail. This is because the weight decay is so small. If you change it to 0.1, it has enough of an effect to make the test fail unless you pass it into torch.optim.SGD.

axiomizer commented 1 year ago

Incorporate weight_decay into test_full_net with commits 69a282b11a443adb6d63cbe06b354e50b73fc4d4 and a49326de28b4f47a8a3339220d82b6acb4c4bcd5

axiomizer commented 1 year ago

Ended up writing a weight decay test, included incidentally in commit 270b8c424a60984d6e9ef706f50c4efe79cbd085 and improved in commit 773db595507d82c58191d4e43c5edb10daf2f22e