Discontinuous Functions

[betanalpha]

What should we do with functions without proper derivatives? Stan includes them and ignores the discontinuities, but this does lead to unintended problems when people try to autodiff through them.

http://www.cplusplus.com/reference/cmath/

• [x] ceil
• [x] floor
• [x] fmod
• [x] trunc
• [x] round
• [ ] lround
• [ ] llround
• [ ] rint
• [ ] lrint
• [ ] llrint
• [ ] nearbyint
• [ ] remainder
• [ ] remquo
• [x] fdim
• [x] fmax
• [x] fmin
• [x] fabs
• [ ] abs
• [x] operator_equal_to
• [x] operator_greater_than_or_equal_to
• [x] operator_greater_than
• [x] operator_less_than_or_equal_to
• [x] operator_less_than
• [x] operator_not_equal_to
• [x] operator_unary_not

[bob-carpenter]

It's not just Stan --- what do you do with the C++ functions that operate on double values if you're building a complete auto-diff system?

Are all the Stan developers on this list? I'd like to include everyone in a discussion this major.

• Bob

On Jul 2, 2014, at 2:45 PM, Michael Betancourt notifications@github.com wrote:

What should we do with functions without proper derivatives? Stan includes them and ignores the discontinuities, but this does lead to unintended problems when people try to autodiff through them.

http://www.cplusplus.com/reference/cmath/

• ceil • [ ] • [ ] • [ ] — Reply to this email directly or view it on GitHub.

[betanalpha]

I'll create a separate thread but formally we might make difference choices for the two systems so I'm keeping this one open here as well.

[syclik]

My opinion is that if Nomad is going to be standalone, we should provide as much functionality as possible. This means doing making decisions based on auto diff and not based on its applications, like HMC.

So, I'm going to suggest we keep all of the functions listed. Perhaps a compromise could be to set an error flag with "discontinuous function, auto diff not valid" flag that could be queries at the end of reach computation. The burden would then be on the user to check the flag.

[betanalpha]

My opinion is that if Nomad is going to be standalone, we should provide as much functionality as possible. This means doing making decisions based on auto diff and not based on its applications, like HMC.

So, I'm going to suggest we keep all of the functions listed. Perhaps a compromise could be to set an error flag with "discontinuous function, auto diff not valid" flag that could be queries at the end of reach computation. The burden would then be on the user to check the flag.

I am also trying to make an autodiff-only decision, which is why I’m happy not supporting functions that work fine for doubles. It is my opinion that we should not be including functions whose derivatives are not defined everywhere.

I’d also prefer to not make any checking optional. Maybe have an option to turn it off, but I don’t want to make it easy for users to mess up without thinking.

[syclik]

I would find the library extremely frustrating and not easy to use if a lot of functions that I'm used to in normal C++ don't exist and I have to find workarounds.

Lots of people I've met want to port old code. I understand that you want to limit misuse, but that would really hinder the usage of the library. On Jul 3, 2014 6:43 AM, "Michael Betancourt" notifications@github.com wrote:

My opinion is that if Nomad is going to be standalone, we should provide as much functionality as possible. This means doing making decisions based on auto diff and not based on its applications, like HMC.

So, I'm going to suggest we keep all of the functions listed. Perhaps a compromise could be to set an error flag with "discontinuous function, auto diff not valid" flag that could be queries at the end of reach computation. The burden would then be on the user to check the flag.

I am also trying to make an autodiff-only decision, which is why I’m happy not supporting functions that work fine for doubles. It is my opinion that we should not be including functions whose derivatives are not defined everywhere.

I’d also prefer to not make any checking optional. Maybe have an option to turn it off, but I don’t want to make it easy for users to mess up without thinking.

— Reply to this email directly or view it on GitHub https://github.com/stan-dev/nomad/issues/2#issuecomment-47891622.

[bob-carpenter]

On Jul 3, 2014, at 8:11 AM, Daniel Lee notifications@github.com wrote:

I would find the library extremely frustrating and not easy to use if a lot of functions that I'm used to in normal C++ don't exist and I have to find workarounds.

Lots of people I've met want to port old code. I understand that you want to limit misuse, but that would really hinder the usage of the library.

I think we all agree that this'll block some uses of the library --- for example, most of Eigen and Boost wouldn't compile. For example, we wouldn't have been able to build prototype autodiff the integrator approach to solving ODEs or any of our initial matrix functions. Users expect these things because the autodiff literature is full of them.

[Let's just take it as read that the vast majority of the scientific literature is written by dimwitted folk who never properly mastered mathematics or computation.]

The difference of opinion is that Michael thinks this is a feature whereas Daniel and I consider it a bug.

As to Daniel's suggestion, I do not like raise-an-error flag approaches.
It's problematic in multi-threaded situations and also a pain to program against. It's a holdover from the C days before exceptions. What I'd prefer if we want some adjustable wrench kind of thing is a static control of what's allowed and what's not --- something like an ENFORCE_STRICT_DIFFERENTIABILITY flag from the outside and a singleton static config as in the Boost error handling.

I'm sympathetic to both sides of this whole argument, but I tend to be more of a Volvo guy, being wary of both cobbled-together dune buggies or very strictly toleranced Formula I cars. To complete this analogy, I think Andrew's on the dune buggy side and Michael on the Formula I side.

I should write a "what kind of programmer are you" blog post. Something along the line of Neal Stephenson's Tank / Batmobile / Ford Taurus essay on operating systems. Now if only I could write as well as he does...

• Bob

[betanalpha]

Attended to in #11.