Open merlinpan opened 5 years ago
You could try training it on cloud TPUs (as in the exact ones that Deepmind used to train AlphaZero and its predecessors) as they are now readily available on Google Cloud (if you have some cash on you ofc) or on Google Colab for free (though there is a certain time limit before the session stops, which is quite annoying). The edge TPUs that you were pointing out to are more suited towards inference on low-power devices.
LZ is based on opencl, you have to rewrite a program to take advantage of tpus.
Ah, that's quite unfortunate. I mean, it wouldn't hurt to try rewriting it as a hobby project, but it would greatly increase the performance of leela-zero considering that you have both the implementation (which could be modified for performance) as well as the hardware. I hear that Google released their next-gen TPUs on the cloud, so in theory leela-zero may be able to surpass AlphaZero in performance.
I suspect with one of these things https://coral.withgoogle.com/ it may actually run at a respectful speed even with a raspberry pi.
Unfortunately, linux only for now.