GRU training time larger with GPU compared with CPU

[georgemavrakis-wings]

Good afternoon,

There seems to be an issue with the training time of the EDDL GRU models for UC1, when using GPU. For instance:

The average training time per epoch of our recurrent model when the training is executed in CPU is 0.26 seconds. On the contrary, when the training is executed in GPU, the average training time is increased to 1.21 seconds.

We have run similar experiments with the pytorch library and the results are the following:

Pytorch GRU model using CPUs: 0.47sec Pytorch GRU model using GPU (NVIDIA RTX 3070): 0.23sec

In the pytorch case, the training time is lower with GPU than CPU, as expected. However, this is not the case with EDDL.

The number of samples used for training is approximately 1,200.

We would like to ask you, if you have any idea about the larger GPU training time?

Thank you in advance,

George Mavrakis

[chavicoski]

Hi,

I have run some tests and I am getting similar results. I am going to see what is happening.

Thanks!

[chavicoski]

Hi George,

We have done some tests and we think that is working properly. When we use a network that is not very small we see a big speed up in GPU (e.g., GPU=0.32s vs CPU=8.70s). And we have only seen some problems like the one you show when using very small networks, problably because of the overhead of using the GPU and moving the data.

Looking at the times that you provide I guess that the network that you are using is very small. And that can be the problem.

If you want share the network topology that you are using and which is the shape of the input, and we can test that concrete example.

[georgemavrakis-wings]

Thank you @chavicoski for your response. We suspected that it might be the case, but we wanted to exclude that there was something from your side.

Best, George