adjoint method costs about 5x memory when solving adjoint equations

[cam1681]

Hi, I reimplemented the code and found that the adjoint method cost about 5x memory when starting solving adjoint equations. And with direct propagation using dpm solver with about only 10 steps, the memory only increases about 2x, so, I am wondering if you also encounter such problem.

[douhongkun]

Hi, I reimplemented the code and found that the adjoint method cost about 5x memory when starting solving adjoint equations. And with direct propagation using dpm solver with about only 10 steps, the memory only increases about 2x, so, I am wondering if you also encounter such problem.

Hello, have you reproduced the security audit experiment yet? I'm running with the source code and the image is all black.

[jiachunp]

Hi, I reimplemented the code and found that the adjoint method cost about 5x memory when starting solving adjoint equations. And with direct propagation using dpm solver with about only 10 steps, the memory only increases about 2x, so, I am wondering if you also encounter such problem.

Hello, have you reproduced the security audit experiment yet? I'm running with the source code and the image is all black.

The main problem is the Neural ODE solver, when I choose the solver type to be Euler and Midpoint, it works. See below.

For the reason why explicit-adams not work, I think it is mainly because the order is too high. In default torchdiffeq, the min order = 4 and the max order = 12 for explicit adams. I usually reset it to min order = 2 and max order = 4, and it also works.

[douhongkun]

@jiachunp Thank you for your timely response, this solves my problem!