Closed Alexivia closed 7 years ago
Regular Caffe does not handle lower or variable precision, but check out Ristretto!
Thanks @shelhamer In the mean time I already moved towards to TensorFlow and I managed to emulate low-precision training.
Happy to hear you found a way forward for your project—if you ever compare to Ristretto it might be interesting to hear about any differences (but on the mailing list, and not the issue tracker). Good luck with your research 🔬
What is the idea behind binarized weights? At first I though I would speed up the forward time, so I checked Ristretto.
But according to the owner:
"Ristretto's purpose is to quantize existing CNNs. Ristretto simulates a hardware accelerator with reduced word width. Ristretto is (slightly) slower than Caffe since the simulation of reduced word width adds overhead."
Reading their paper I kind of get the idea of simulation of reduced word width for running Caffe on low power embedded devices.
With this in mind is it actually possible to speed up the CPU forward time?
Issue summary
Recent developments have shown good results of Binarized Neural Networks, which are a new kind of CNN that works with binary weights instead of the typical 32-bit floats (refer to this paper). To train these BNNs they use Theano. Is there any way I can make a Caffe layer type that works with 1-bit weights instead of 32-bit FP?
Your system configuration
Operating system: Fedora 21 Compiler: gcc (GCC) 4.9.2 20150212 CUDA version (if applicable): V7.5.17 BLAS: blas.x86_64 3.5.0-10.fc21 Python: python.x86_64 2.7.8-15.fc21