CPU and GPU gave different results

torch / cunn

Other

215 stars 174 forks source link

CPU and GPU gave different results #464

Open hainguyenct opened 7 years ago

hainguyenct commented 7 years ago

Hello everyone, I received different results when I tried to train with GPU and CPU. For using CPU, I got accuracy of 87% while GPU got 82%. Does anyone has met this situation? Thank you,

elgalu commented 5 years ago

Can you double check?

[ ] all training data is identical
[ ] all stochastic parts of training use the same random seed and generator
[ ] all data types are identical precision (e.g. all vectors and matrices are 32-bit or 64-bit floats)

It could be a difference in the execution order of the parallelizable operations following floating point error propagation: https://discuss.pytorch.org/t/why-different-results-when-multiplying-in-cpu-than-in-gpu/1356/4