search

stan-dev / math

The Stan Math Library is a C++ template library for automatic differentiation of any order using forward, reverse, and mixed modes. It includes a range of built-in functions for probabilistic modeling, linear algebra, and equation solving.
https://mc-stan.org
BSD 3-Clause "New" or "Revised" License
750 stars 189 forks source link

test fixture test_gradients problem #102

Open syclik opened 9 years ago

syclik commented 9 years ago

From @rtrangucci on January 21, 2015 1:15

the test_gradients test in test_fixture_distr.hpp, test_fixture_cdf.hpp, test_fixture_cdf_log.hpp and test_fixture_ccdf_log.hpp is broken.

The test_gradients in test_fixture_distr.hpp is intended to test a density's implementation of partials with an autodiffed naive implementation of the density. However, due to the current pattern, the test always passes. To verify this, change line 87 in stan/src/test/unit-distribution/univariate/continuous/normal/normal_test.hpp from:

return -0.5 * (y - mu) * (y - mu) / (sigma * sigma)

to:

return -0.5 * (y - mu) * (y - mu) / (sigma * sigma) * 100

and run:

./runTests.py -j2 src/test/unit-distribution/univariate/continuous/normal/normal_00000_generated_v_test.cpp

noting that the "Function" tests for v32 and v36 pass when they shouldn't. The problematic section in test_fixture_distr.hpp are lines 522-552:

      Scalar0 p0 = get_param<Scalar0>(parameters[n], 0);
      Scalar1 p1 = get_param<Scalar1>(parameters[n], 1);
      Scalar2 p2 = get_param<Scalar2>(parameters[n], 2);
      Scalar3 p3 = get_param<Scalar3>(parameters[n], 3);
      Scalar4 p4 = get_param<Scalar4>(parameters[n], 4);
      Scalar5 p5 = get_param<Scalar5>(parameters[n], 5);

      vector<var> x1;
      vector<var> x2;
      vector<var> x3;
      vector<var> y1;
      vector<var> y2;
      vector<var> y3;
      add_vars(x1, p0, p1, p2, p3, p4, p5);
      add_vars(x2, p0, p1, p2, p3, p4, p5);
      add_vars(x3, p0, p1, p2, p3, p4, p5);
      add_vars(y1, p0, p1, p2, p3, p4, p5);
      add_vars(y2, p0, p1, p2, p3, p4, p5);
      add_vars(y3, p0, p1, p2, p3, p4, p5);

      T_return_type logprob = TestClass.template log_prob
        <Scalar0,Scalar1,Scalar2,Scalar3,Scalar4,Scalar5>
        (p0,p1,p2,p3,p4,p5);

      T_return_type logprob_funct = TestClass.template log_prob_function
        <Scalar0,Scalar1,Scalar2,Scalar3,Scalar4,Scalar5>
        (p0,p1,p2,p3,p4,p5);

      calculate_gradients_1storder(expected_gradients1, logprob_funct, x1);
      calculate_gradients_1storder(gradients1, logprob, y1);

The reason the tests pass is because calculate_gradients_1storder in stan/src/test/unit-distribution/test_fixture_distr.hpp calls stan::agrad::recover_memory() and the naive implementation and the density function are called with the same vars. y1 and x1 point to the same varis, whose adjoints are updated when calculate_gradients_1storder is called the first time.

The fix is to change the order of how calculate_gradients is called and how vars are created for the naive implementation vs. the stan::prob densities within test_gradients.

Copied from original issue: stan-dev/stan#1224

bob-carpenter commented 7 years ago

This seems like a major issue. @rtrangucci --- it sounds like you figured out how to fix this --- would you mind putting the fix in the form of a pull request? Or saying a bit more about what you mean by "change order". I'm happy to do the mechanical bits, but that's an incredibly complicated fixture. I'm upping the due date to the next release.