lightvector
commented
4 years ago Thanks for the interest. Good questions.
It seems that in the new version of the KataGo paper, all the details about board masking to handle different board sizes in the same mini-batch are removed.
It was removed from the main paper in the recent versions just due to space limitations and because it wasn't a focus of the research. The method has remained unchanged, and the actual architectural details are still present and precisely described in the appendix.
I was thinking of using the same board sizes in one mini-batch but alternate between different board sizes from minibatch to minibatch. This because for one it avoids the use of masks. Keras, for instance, accepts inputs of (None, None, C) instead of (B, B, C) and handles fully-convolutional network training on variable board sizes.
You could certainly make the board sizes vary by batch instead of by sample. The tradeoffs I can think of would be:
- Data generation and shuffling become slightly more complex since you have to keep different data separated and in the right ratios.
- The training performance improves a little since sometimes you're operating on smaller tensors some of the time and so this will make you run faster, (but not necessarily improving self-play performance, which dominates the overall cost)
- The training probably becomes a little worse you have much heavier correlation between samples within batches. In particular, gradient steps along the "features-that-care-about-board-size" axis of the loss landscape might be as much as sqrt(batch_size) times larger than before, or something like that, giving you jumpier gradients.
Is the global-pooling structure somewhat similar to squeeze-excitation nets? Do you think SE block needs to be modified similarly when training with multiple board sizes?
Yes, SE would also need to be modified.
Also, I don't understand why out of the 3C channels the global-pooling structure produces for the value-head, 2C channels are board sized scaled versions of the C channels. Since a dense layer follows, the NN should be able to learn to scale appropriately with the board size.
Think about this. Suppose the pooling only reports the mean activation of each channel. Suppose that for some of the channels actually the more useful feature that future layers "want" is the mean activation of these channels scaled by board size, rather than just plain mean.
Now, what is the dense layer supposed to do? What fixed linear combination of means-of-channels gives you mean-scaled-by-board-size? (Let's also suppose that the activation strengths of the channels, which were computed via a few convolutions in earlier layers, don't vary much with board size in the first place - the entire point of pooling at all is because convolutions are bad at this even if they can do it somewhat. Perhaps even this is early enough in the net that the perceptual radius is literally too small).
Also, wouldn't it be easier to just provide an input plane or single binary feature indicating the relative size of the board?
You could do this. Mechanically then the way the net would be able to scale different values internally by board size would be to break up board size piecewise into a bunch of fragments using ReLU, take the to-be-scaled-value and split it across multiple channels with different weights based on which board size fragments are active, and then recombine with different weights. (Recall that in a convolutional net, there are no product terms between inputs. There are only products between inputs and fixed weights, so any scaling of inputs by each other must be done by defining piecewise linear chunks through multiple channels).
Intuitively I'd guess scaling the pooling directly to be better than forcing the net to do that. Besides, if the neural net actually did also "want" board size as a numeric input feature, after merely one such global pooling layer it can trivially compute that on its own and then use it as desired in all subsequent layers, so scaled the pooling should almost just dominate an input feature.
Another question is that I used global-max-pooling in the value head by mistake and noticed the net has difficultly learning. Did you observe similar behavior?
No, I never even tested such a thing in the first place because I had a strong intuition from first principles that this would be a bad idea.
To first order, to find chance that you win, given some uncertainty in one's evaluation, you need to do something like expected score / sqrt(variance of score). The units could be arbitrary and not simply "points" and with hacks for discreteness of situations and correlations between fights, etc. Still, this is the natural first-order thing. Expected score is the sum over the board of the score from each part of the board. Variance of score is going to look like a sum over the board of the variance contribution from each part of the board due to how "uncertain" you think your evaluation of that part of the board is. The neural net may of course choose to do some of this scaling before pooling, and will need lots of adjustments to deal with correlations and other complexities.
But either way, you want to sum across the board, not max across the board.
The same thing is even more obviously true for predicting the score itself - you want to sum the territory across the board, not max it.
Sums can also be computed by scaling a mean pooling, so mean pooling is also okay as the "basic" operation . But: the same number of total points advantage on a large board translates to a smaller advantage in winning chance, because more situations means that even holding constant the uncertainty of each individual situation, the total uncertainty will be larger. Therefore winrate and score estimation will want different scalings by board size. So in addition to simply max alone being bad, there should also be a benefit to directly providing different board-size scalings of the pooling in the value head.
dshawul
l1t1
This is not an issue but a question about how katago handles different board sizes. Please feel free to move it or direct me to where to post the question if it can't stay here.
It seems that in the new version of the KataGo paper, all the details about board masking to handle different board sizes in the same mini-batch are removed. I was thinking of using the same board sizes in one mini-batch but alternate between different board sizes from minibatch to minibatch. This because for one it avoids the use of masks. Keras, for instance, accepts inputs of (None, None, C) instead of (B, B, C) and handles fully-convolutional network training on variable board sizes.
Is the global-pooling structure somewhat similar to squeeze-excitation nets? Do you think SE block needs to be modified similarly when training with multiple board sizes? Also, I don't understand why out of the 3C channels the global-pooling structure produces for the value-head, 2C channels are board sized scaled versions of the C channels. Since a dense layer follows, the NN should be able to learn to scale appropriately with the board size. Also, wouldn't it be easier to just provide an input plane or single binary feature indicating the relative size of the board? Another question is that I used global-max-pooling in the value head by mistake and noticed the net has difficultly learning. Did you observe similar behavior?
Thanks for the great work!