search

rtqichen / torchdiffeq

Differentiable ODE solvers with full GPU support and O(1)-memory backpropagation.
MIT License
5.61k stars 930 forks source link

runtime of ode_demo.py using adjoint vs. not using it #243

Open matibilkis opened 1 year ago

matibilkis commented 1 year ago

Hello!

I became super interested in this project! Thanks a lot for pushing this forward.

I noticed that while running ode_demo.py using odeint_adjoint is slower than naively integrating using Euler and backprop...

is there any explanation for this ? I understood from your paper that this should be expected to be faster...

Thanks!

shiqingw commented 1 year ago

Hi @matibilkis, I recently also tested odeint and odeint_adjoint on my own problem and found out that odeint_adjoint did not finish overnight while odeint finished within 1h20min. I could be wrong in my explanation, but what I learned from README.md is that odeint_adjoint is memory efficient but not necessarily faster. In the paper (Algorithm 2), this is done by augmenting the dimension of the original dynamics. In my problem, the original dynamics is 2-dimensional but the total number of my parameters is over 50k. Therefore, odeint is solving a 2-dimensional ode, while odeint_adjoint is solving a (2+50k)-dimensional ode (maybe plus some other augmented states). I am also curious about how to use odeint_adjoint correctly and what kind of computational time I should expect.