PZ Issues 5

[Klazkin]

Explain the context of the issue, what is being addressed in detail.

The goal

• [x] Setup another testing routine...
• [x] Shacles child represented differently.
• [ ] Random sampling is done towards the start of the game.
• [x] Learning rate scales over training steps - highly positive impact (accuracy 0.49 -> 0.52)
  • increasing to 128 filters further improved accuracy to 0.60
• [ ] Dynamic number of training steps / end early call back with save best
• [x] Try Input Normalization layers
• [x] Alternative Architecture for Policy
  • [x] Hybrid architecture
  • [x] Multi-Head
• [x] Focal Loss, various loss functions
• [x] Additional Batch Normalization in Policy head
• [ ] ConvNext/GeLU instead of ReLU + ResBlock.
• [ ] LN vs BN
• [ ] Weight Standartization + GN
• [ ] ResNet Encoder
• [x] Try Excluding samples with low visits count.
• [x] Clean up self-play output.
• [ ] Softmax Scoring on MCTS generated data.
• [ ] Impact of kernel initializers.
• [x] implement data augmentation: board flipping along X and Y axis
• [ ] Impact of re-introducing bias weights.

Time tracking

Time Estimate: 0 hours 0 minutes Time spent: 8 hours 40 minutes

Resources

...