search

BVLC / caffe

Caffe: a fast open framework for deep learning.
http://caffe.berkeleyvision.org/
Other
34.09k stars 18.7k forks source link

same input, different output. precision difference between multiple running. #3943

Open turinglife opened 8 years ago

turinglife commented 8 years ago

Have you guys encountered the following problem.

There are the precision differences of outputs of the same parallel interface with same input values on multiple being invoked. precision differences between two calling is randomly appear.

Anyone can tell me why it is?

Thanks so much.

The following is an example. The parallel interface Concat was employed on this experiment, which had the same input values such as top_diff etc. However, Concat randomly got the precision difference between two running.

environment: Ubuntu 14.04 X86_64, both CUDA7.0 and CUDA7.5 were employed separately, cuDNN V3, the same implementation of parallel interface Concat as the one in the current master branch.

333

333

turinglife commented 8 years ago

@shelhamer, thanks in advance.