kblomdahl
commented
6 years ago All these enhancements are for the policy network, which is generally doing pretty well even without these tweaks. I would be more interesting in seeing if this improves the value head, in which you can observe the following issue consistently:
- It has issues identifying whether large dragons are alive or dead.
Following the logic posted in lightvector's repository one could reasonably expect Parametric ReLUs and Global Pooling Properties to have little effect on this problem. However Chain Pooling might greatly help with this issue.
To help with the performance impact, one could limit the chain pooling to the final convolutional layer in the value head with input x:
y₀ ← max chain pooling of xy₁ ← relu(bn(C(W₁, [x y₀])) + b₁)y₂ ← relu(W₂ y₁ + b₂)y₃ ← tanh(W₃ y₂ + b₃)
This is only one layer of chain pooling so we expect it to have issues identifying problems with loosely connected groups or certain types of false eyes. But it would probably help in most situations.
To implement this we would have to do a custom CUDA kernel, which could probably be done relatively easily using some flood filling technique:
int global_index = threadIdx.x;
do {
if (global_index == 0) {
*is_dirty = 0;
}
__syncthreads(); // barrier
for (int i = 0; i < batch_size; ++i) {
for (int c = 0; c < num_channels; ++c) {
const float original = data[c, global_index];
if (chain[global_index] == chain[N[global_index]])
data[i, c, global_index] = max(data[i, c, global_index], data[i, c, N[global_index]]);
if (chain[global_index] == chain[E[global_index]])
data[i, c, global_index] = max(data[i, c, global_index], data[i, c, E[global_index]]);
if (chain[global_index] == chain[S[global_index]])
data[i, c, global_index] = max(data[i, c, global_index], data[i, c, S[global_index]]);
if (chain[global_index] == chain[W[global_index]])
data[i, c, global_index] = max(data[i, c, global_index], data[i, c, W[global_index]]);
if (original < data[c, global_index])
*is_dirty = 1;
}
}
__syncthreads(); // barrier
} while (*is_dirty > 0);
lightvector
killerducky
alreadydone
Check out some of ideas mentioned here to enhance the neural network:
@lightvector https://github.com/lightvector/GoNN
Initial thoughts on the concepts without any further research to back it up:
Chain Pooling This probably gives worse results since the pooling destroy the local shape information. Might be interesting to do a dense block approach where each residual blocks becomes a DenseNet [1]. This way each residual block would benefit from both the pooling and the local shape:
xcompute the the chain pooling of each channel and store the result inyₛ.xandyₛchannel-wise intoyₓₛ(soyₓₛhas shape[256, 19, 19]).y₁ <- relu(C(W₁, yₓₛ) + b₁)(y₁has shape[128, 19, 19])y₁andyₛchannel-wise intoy₁ₛ(soy₁ₛhas shape[256, 19, 19]).y₂ <- relu(C(W₂, y₁) + x + b₂)y₂I suspect this is too expensive to do in practice (no good support for it in cuDNN), but it is a very interesting idea.