SBuercklin
commented
3 years ago Lowering precision should be possible as it stands. You need to use Float32 instead of Float64 for all of your values throughout. It's possible that some numbers were left as Float64, in which case they would fall back to using the slower double precision arithmetic.
As for how significant a speedup you'll see, I'm not sure what to expect on that front. I haven't spend much time trying to benchmark the operations against each other.
GuillaumeLam
This is a further objective, however, allowing for lower precision of weights could allow for faster running of networks while maintaining precision. I tried lowering the precision of a small network and saw barely any speedup, but a bigger network could see more of a speedup. Furthermore, on smaller memory sized hardware, a lower precision network could decide btwn if the network runs or fills up the whole memory.