Closed GuillaumeLam closed 3 years ago
Lowering precision should be possible as it stands. You need to use Float32
instead of Float64
for all of your values throughout. It's possible that some numbers were left as Float64
, in which case they would fall back to using the slower double precision arithmetic.
As for how significant a speedup you'll see, I'm not sure what to expect on that front. I haven't spend much time trying to benchmark the operations against each other.
What do you think about lower precision (ie float8/16)?
I guess this is quite a far off improvement tbh so I think I'll close the issue but I did want to get a feel for how plausible this would be.
This is a further objective, however, allowing for lower precision of weights could allow for faster running of networks while maintaining precision. I tried lowering the precision of a small network and saw barely any speedup, but a bigger network could see more of a speedup. Furthermore, on smaller memory sized hardware, a lower precision network could decide btwn if the network runs or fills up the whole memory.