Non-linear slowdown calling grad repeatedly

[tonigi]

Description

I came across an unexpected slow-down when repeatedly computing the gradient of a function. The calls are independent and should leave no state behind, but the compute time grows more than linearly. If it's the function tree growing, I don't see why or how to stop it between calls. The same does not occur when rewriting as a functor.

Example

#include <stan/math/rev/mat.hpp>

using std::vector;

using stan::math::var;
using stan::math::vector_v;

void test(bool print=true)
{
    vector_v x(3);
    x << 1 ,          0 ,           0;

    var y = dot_self(x);
    y.grad();

    if(print) {
    std::cout << "y: " << y.val() << std::endl;
    for (int i=0; i<3; i++) std::cout << x[i].adj() << " ";
        std::cout << std::endl;
    }

    stan::math::set_zero_all_adjoints();

}

int main(int argc, char ** argv)
{
    test();
    for(int i=0; i<atoi(argv[1]); i++)
    test(false);
}

Expected Output

Output is ok. The runtime is not.

$ time ./stan_perf_minimal 10000
y: 1
2 0 0

real    0m1.042s
user    0m1.016s
sys 0m0.016s

$ time ./stan_perf_minimal 20000
y: 1
2 0 0

real    0m4.368s
user    0m4.313s
sys 0m0.032s

$time ./stan_perf_minimal 30000
y: 1
2 0 0

real    0m11.002s
user    0m10.769s
sys 0m0.063s

Current Version:

v2.19.1

[wds15]

You need to recover memory between repetitions otherwise the ad tree size grows with each iteration. Please close this if that fixes your problem

[tonigi]

Thanks a lot. Solved by stan::math::recover_memory()