Why is Neural Ode training faster in CPU than in GPU?

DiffEqML / torchdyn

A PyTorch library entirely dedicated to neural differential equations, implicit models and related numerical methods

https://torchdyn.org

Apache License 2.0

1.35k stars 125 forks source link

Closed bigyaan closed 2 years ago

bigyaan commented 2 years ago

I am trying to train a Neural ODE in my local gpu to speed up my training. However, for my surprise it slowed the training rather than speeding it. I have a nividia gtx 1060ti gpu with 4gb ram. I have shared my code in given link below: https://colab.research.google.com/drive/1DXFkPzOPeqBBmYON3TOtxi9tHwrUmgBs?usp=sharing

joglekara commented 2 years ago

Hey, we discussed this a bit over Slack so I'm closing this. Feel free to reopen if you would like to restart the discussion