Speed of training and related computer specs

hx173149 / C3D-tensorflow

C3D is a modified version of BVLC tensorflow to support 3D ConvNets.

MIT License

588 stars 262 forks source link

Speed of training and related computer specs #62

Open JulioEndara opened 6 years ago

JulioEndara commented 6 years ago

When training each step it's taking a little over 320 seconds. At that rate for 5000 steps it'd take over 18 days. I'm using a VM in google cloud with the following specs: 8vCPUs, 52Gb RAM, 1GPU (NVIDIA Tesla P100).

Is that the expected speed for that hardware? is there anything to tune in the code to take full advantage of the hardware? Can you share your specs and the speed you are getting?

thanks!

Mohamad73 commented 6 years ago

I am doing it on two GTX 1070 and it is taking around 1.5 second per step

Nortinwell commented 6 years ago

Maybe the TensorFlow you installed is not the GPU version, try pip install tensorflow-gpu insteal of tensorflow. And maybe you should change the "gpu_num" in train_c3d_ucf101.py