Remove fvar<double> and fvar<fvar<double>> tests

[seantalts]

Summary:

We don't actually care about forward mode autodiff with doubles under the hood; we only use the forward mode system for higher order autodiff (and reverse mode vars under the hood). We discussed in one of the Stan meetings that we should be able to remove all of the fd and ffd probability distribution tests, which I estimate will cut down the distribution test time from ~6 hours to ~3.5 hours.

Current Version:

v2.15.0

[bob-carpenter]

We do care about fvar<double> and fvar<fvar<double> > for the math library.

If you're just looking at what's available through the interfaces, that's only reverse mode.

I'm starting a pass through the math lib and will generalize the tests. I will try to make them configurable so you can turn on/off different instantiations in the tests.

I don't mind testing these more intermittently, but I don't want to get rid of them. As I was saying at the board, they're useful. At the very least fvar<double> is going to be the most efficient way to compute Jacobians once we get fvar tested thoroughly enough that we trust it.

[seantalts]

We're planning to one day use forward mode to compute Jacobians? In what context?

[betanalpha]

You mean directional derivatives? That’s all fvar will give you.

On May 17, 2017, at 12:00 PM, Bob Carpenter notifications@github.com wrote:

We do care about fvar and fvar<fvar > for the math library.

If you're just looking at what's available through the interfaces, that's only reverse mode.

I'm starting a pass through the math lib and will generalize the tests. I will try to make them configurable so you can turn on/off different instantiations in the tests.

I don't mind testing these more intermittently, but I don't want to get rid of them. As I was saying at the board, they're useful. At the very least fvar is going to be the most efficient way to compute Jacobians once we get fvar tested thoroughly enough that we trust it.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/stan-dev/math/issues/555#issuecomment-302137384, or mute the thread https://github.com/notifications/unsubscribe-auth/ABdNlgoM9vko_bdqmoeh7GaWMl8UB2dyks5r6xmwgaJpZM4NeDpK.

[bob-carpenter]

Inside the ODEs, we use nested reverse mode in a couple places to compute Jacobians (need them for system w.r.t. parameters derivatives to compute the coupled solver and for the stiff solver to compute that and the system Jacobian for controlling steps).

• Bob

On May 17, 2017, at 12:11 PM, seantalts notifications@github.com wrote:

We're planning to one day use forward mode to compute Jacobians? In what context?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[bob-carpenter]

The important thing is that it gives you multivariate directional derivatives.

If you have f:R^N -> R^M, then

• one pass of forward mode gives you d/dx[n] f(x)[m] for a single n in 1:N and all m in 1:M
• N passes of forward mode gives you the entire N x M Jacobian

This can be generalized to give you directional derivatives, but I don't think we ned that in the ODE solver.

In contrast,

• one pass of reverse mode gives you d/dx[n] f(x)[m] for a single m in 1:M and all n in 1:N.
• M passes of reverse mode gives you the entire Jacobian

We should really write up forward-mode and higher-order autodiff!

[betanalpha]

Currently the ODE solver uses reverse mode for computing the system Jacobian.

Are you arguing that you want to preserve fvar in case we want to compute more Jacobians with M >> N or even replace the current ODE Jacobian code with forward mode?

On May 17, 2017, at 2:26 PM, Bob Carpenter notifications@github.com wrote:

The important thing is that it gives you multivariate directional derivatives.

If you have f:R^N -> R^M, then

• one pass of forward mode gives you d/dx[n] f(x)[m] for a single n in 1:N and all m in 1:M
• N passes of forward mode gives you the entire N x M Jacobian

This can be generalized to give you directional derivatives, but I don't think we ned that in the ODE solver.

In contrast,

• one pass of reverse mode gives you d/dx[n] f(x)[m] for a single m in 1:M and all n in 1:N.
• M passes of reverse mode gives you the entire Jacobian

We should really write up forward-mode and higher-order autodiff!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/stan-dev/math/issues/555#issuecomment-302185300, or mute the thread https://github.com/notifications/unsubscribe-auth/ABdNlo_-0XmLu6O4Zb3MkAdDYACwiQRcks5r6zvmgaJpZM4NeDpK.

[bob-carpenter]

Not so much arguing as conjecturing that forward mode would be a lot faster. It's purely compiled, so it has a huge advantage. I'll test that, of course.

But like I said, I'm OK with option (2), which is turning off all the forward-mode testing on our normal pull requests. I'm working right now on adding a general forward-mode test framework, but we could isolate it somehow if you think it's worth it for speeding up general testing.

Are you arguing that you want to preserve fvar in case we want to compute more Jacobians with M >> N or even replace the current ODE Jacobian code with forward mode?

[bob-carpenter]

I'm closing because we don't want to do this.