Closed whria78 closed 5 years ago
FP16 model : always slow FP32 model : If I set default_backward_math from FLOAT to FLOAT16, it shows the slowness.
If I change FLOAT16 to FLOAT in the deploy.prototxt of the FP16 model, the FP16 model recover the speed. But the use of FLOAT on the FP16 model does not look appropriate.
default_forward_math : FLOAT default_backward_math : FLOAT16
Hi @whria78 seems like not all layers you use are implemented for GPU. Unfortunately, FLOAT16 runs very slow on CPU, this is expected.
@drnikolaev Thank you for the reply.
As you said, I changed "engine: CAFFE" to "engine: CUDNN". But it also make the problem.
And Today, I found that this problem occur only with Pascal GPUs (1080ti, 1050ti, 1070).
I tested my 2080 GPU, there was no problem.
Is there any chance that there may be some problem in checking GPU type for the special acceleration of Volta GPU?
@whria78 engine: CAFFE
actually stands for GPU if implemented, CPU otherwise
whereas engine: CUDNN
stands for GPU + CUDNN
.
As of different nets - try to increase batch size up to the limit and notice some of them getting hot (depending on data rate consumed). Fore example, old good AlexNet is the most data hungry.
Environment
Ubuntu 16.04 1050ti and 1080ti CUDA 10.1 / cudnn 7.5 Test code and pretrained Model https://1drv.ms/u/s!Al4S1efzlWvXh7hQ6eD4B530c8UCKA?e=2cOvxV
Problem FP16 training : nvcaffe 0.17.2 > FP32 BVLC caffe with axpy layer patch --> faster than BVLC FP16 testing (deploying) : BVLC caffe >>> nvcaffe 0.17.2 (0.2 sec) --> slow 10 times.
I also tested with /nvcaffe/python/classifier.py , but it also slow too.