search

stan-dev / nomad

Fast autodiff.
Other
19 stars 2 forks source link

How To Test Equals Operators? #13

Open betanalpha opened 10 years ago

betanalpha commented 10 years ago

Comparison operators are very bad for autodiff because they admit the construction of discontinuous functions. Equals operators are particularly bad because they can't even be tested with finite differences, as the perturbations cause the comparison to fail. Not sure how these operators would be tested.

bob-carpenter commented 10 years ago

operator= and operator!= should return int values, so I don't see an issue because there's no derivative created.

It's only the context they're used in that can create problems as in

if (a == b) stmt1 else stmt2;

The problem isn't the comparison operator, it's in its usage.

On Aug 9, 2014, at 5:55 PM, Michael Betancourt notifications@github.com wrote:

Comparison operators are very bad for autodiff because they admit the construction of discontinuous functions. Equals operators are particularly bad because they can't even be tested with finite differences, as the perturbations cause the comparison to fail. Not sure how these operators would be tested.

• src/scalar/operators/nonsmooth_operators/operator_equal_to • src/scalar/operators/nonsmooth_operators/operator_not_equal_to • src/scalar/operators/nonsmooth_operators/operator_unary_not — Reply to this email directly or view it on GitHub.

betanalpha commented 10 years ago

The comparison operators return bools, and it’s the usage they admit that is exactly the problem. You can’t differentiate through the comparison analytically or approximately (which is why the finite difference fails) so the very existence of var-valued equals operators is ill-posed. Ill-defined tests are just a side effect of this — I’m not sure if there is a solution.

On Aug 10, 2014, at 7:00 PM, Bob Carpenter notifications@github.com wrote:

operator= and operator!= should return int values, so I don't see an issue because there's no derivative created.

It's only the context they're used in that can create problems as in

if (a == b) stmt1 else stmt2;

The problem isn't the comparison operator, it's in its usage.

  • Bob

On Aug 9, 2014, at 5:55 PM, Michael Betancourt notifications@github.com wrote:

Comparison operators are very bad for autodiff because they admit the construction of discontinuous functions. Equals operators are particularly bad because they can't even be tested with finite differences, as the perturbations cause the comparison to fail. Not sure how these operators would be tested.

• src/scalar/operators/nonsmooth_operators/operator_equal_to • src/scalar/operators/nonsmooth_operators/operator_not_equal_to • src/scalar/operators/nonsmooth_operators/operator_unary_not — Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub.